Limepoint Automation RPC Utils

Eventually this will evolve into a full set of RPC utilities, but for now:

• Pluggable transports, though only SSH is implemented

• Support for hopping through jump hosts

• Sudo with password/nopassword

• Keys or password auth

• Does not rely on Port Forwarding, clean terminals, or so on, so should work on any weird enterprise network

• Reuses existing connections to avoid lots of connectivity latency

Provides * simple shell execution * more complex shell execution with wait/inject functions * file transfers * remote ruby code execution, including shipping a transient interpreter * access to files as ruby objects

Environment Variables

• LAS_PROMPT_TIMEOUT - timeout for how long to wait for a prompt. Default is 300s, to allow for slow sudo/ldap configurations

• LAS_READY_TIMEOUT - the time between executing a script, and seeing the “script started” output. Default is 30s.

• LAS_WAIT_TIMEOUT - the time to wait at the end of the script. defualt is 30s.

• LAS_DISABLE_TRANSFORM - dont use a microservice for reconnectable connections, but instead use a pure SSH connection

Getting static ruby to endpoints

How this happens is controlled by the environment variable LAS_RUBY_METHOD - currnetly:

• gem - push from the gem via SSH (this is the default)

• http - grab from limepoint’s public HTTP server

• any url starting with ‘http’ - grab from that URL specifically

Use

Usually, for the general case, you use this through a transport manager - so you end up with code like:

transport_options = {}
      if @sudo
        transport_options[:user] = @sudo_user
        transport_options[:password] = @sudo_password
        transport_options[:keys] = [@ssh_key]
        transport_options[:sudo_user] = @ssh_username
        transport_options[:port] = @ssh_port
      else
        transport_options[:user] = @ssh_username
        transport_options[:password] = @ssh_password
        transport_options[:keys] = [@ssh_key]
        transport_options[:port] = @ssh_port
      end
$transport_factory = ASRpcUtils::TransportFactory.new(default_type: 'ssh', default_transport_options: transport_options)
#Also now set_options_for_host to do per host options
$transport_factory.set_options_for_host('jjosi1', {user: jj})
# This takes a string
$transport_factory.add_default_path("/oracle/product/bin")
# THis takes a hash
$transport_factory.add_default_env({"ORACLE_HOME" => "/oracle/product")
conn = $transport_factory.transport(hostname)
reply=conn.excute("echo $PATH")
puts reply

For now, see samples in test/ folder, but:

TransportManager

• See above - JJ needs to document this more! But usually, call .transport to get a transport once set up

• Usually you want to set environments on this, unless you really specifically want it on your connection object

Transport

• LASRpcUtils::Transport.new(host, type, transport_options) Initiate a transport Example: LASRpcUtils::Transport.new('testosi1.mintpress.io', 'ssh', { sudo_user: 'oracle', user: 'root', keys: [ '~/.ssh/id_rsa' ]})

• transport.add_hop Add a hop to a transport

• transport.alive? Test if a transport is alive

• transport.execute(command) Execute a remote command. Returns a triple of return_code,stdout,stderr Example: transport.execute("whoami")

• transport.content(filename) Return the content of a file Example: transport.content("/etc/hosts")

• transport.getfile(local,remote)

• transport.putfile(local,remote)

• transport.file_exists?(filename) Boolean if a file/folder exists

• transport.ruby(&block) Execute ruby Example:

  filename="/etc/hosts"
     content=transport.ruby do 
   File.read(filename)
     end
  

The ruby transport will get the contents of all marshalable local variables. Native ruby types will be fine, however un-marshalable objects will NOT be transported - obviously, anything with IO is the big gotcha here. * transport.port_forward_via_local(listen_address_on_transport, listen_port_on_transport, destination_address, destination_port) Sets up a port forward. example: transport.port_forward_via_local(“localhost”, 443, “chef-server.mintpress.io”, 443)

• transport.File Return a remote ruby file object Example: transport.File.open("/etc/hosts").read()

• transport.ruby_require(thing) Require a module in the rmeote ruby session

• transport.ruby_include(thing) Include a namespace in the remote ruby sessioon

• transport.wrap_object(obj) Add a wrapped object - it appears as a function of transport just like the below objects

  transport.ruby_require("rexml/document")
  transport.ruby_include("REXML")
  transport.wrap_object("Document")
  doc=transport.Document.new "/environmint/runtime/data/MintPress/bamboo.cfg.xml"
  puts doc.class
  doc.root.attributes.each do |e|
    puts "ele: #{e}"
  end
  

• transport.Dir

• transport.IO

• transport.TcpSocket

• transport.UdpSocket

• transport.Socket

• transport.HTTPClient

• transport.RestClient

• transport.OCI8

• transport.Tempfile Return remote ruby objects If you need more of these, add them to $wrapped_remote_objects in transport.rb!

Note that there are some limitations due to the way the wrapping works - the most notable being that, while blocks work normally for File/HTTP objects, they do NOT work normally for OCI8 objects, since it uses block detection to behave differently. For objects like this, you should use a ruby block instead - i.e.

transport.ruby do
  db=OCI8.new('sys', 'welcome1', '//uomdb1.mintpress.io/ORADB', :SYSDBA)
  db.exec("select * from dba_users") do |r|
    puts r.join(',')
  end
end

will work, whereas:

db=transport.OCI8.new('sys', 'welcome1', '//uomdb1.mintpress.io/ORADB', :SYSDBA)
db.exec("select * from dba_users") do |r|
  puts r.join(',')
end

Will throw an exception because the object does not take a block

• transport.start_execute(command) start a command runnning on an interactive channel

• channel.send_text(text) send text to the channel

• channel.wait_for_stdout(wtext, type: 'normal', failtext: nil, timeout: 3600, trim_working: false, local_echo: true, filter: ['---limepoint-']) Wait for stdout to do something. Accepts:

• wtext - array of matches

• type - match type. normal, fnmatch, or regex

• failtext - if this match is hit, fail immediatly

• timeout - how long to wait

• trim_working - trim the log. this is generally only for internal use

• local_echo - log as we wait

• filter - items to not show in the logs - use this for escapes and such

• channel.wait_for_complete Wait for your channel to complete. Returns the triplet (returncode, stdout, stderr)

License & Authors

• Author:: LimePoint (support@limepoint.com)

# MintPress® - Automation and Configuration Management
#
# Copyright © 2014 LimePoint. All rights reserved.
#
# This program and its contents are confidential and owned by LimePoint.
# Only licenced users are permitted to access and use of this file.
# This program (or any part of it) may not be disclosed, copied or used
# except as expressly permitted in LimePoint’s End User Licence Agreement.
#
# LimePoint® and MintPress® are Registered Trademarks of LimePoint IP Limited.
# For more information contact LimePoint at http://www.limepoint.com