oracle-otd
Copyright 2019 © LimePoint Pty Ltd. All rights reserved.
Introduction
This controller implements functionality for Oracle Traffic Director. Note that for floating IPs, if the OTD controller is not provided access to the root user, you will need to enable certain sysctl parameters to take advantage of floating IPs in high availability modes - please consult the OTD documentation as to how to ensure these are correctly configured for your environment (specifically, docs.oracle.com/middleware/1221/otd/admin/ha.htm#OTADG297 )
Versions
Supported Versions: 12.2.1.2, 12.2.1.3
Supported Configurations
Currently, the following configurations are supported: * Colocated with a full Weblogic Domain (this requires access to a database) * Colocated with a restricted JRF Weblogic Domain (this does not require access to a database)
Standalone mode is currently not fully supported.
How it works
The OTD controller wraps the provided API by oracle (documented at docs.oracle.com/middleware/12213/otd/OTWST/reference.htm , - knowledge of the OTD product is beyond the scope of this document, however fundamentally an OTD setup consists of a set of configurations, which contain items such as server pools and origin servers. You can see this tree by refering to the refernce documentation for the MintPress::OracleTrafficDirectory::Configuration class.
domain=MintPress::OracleWeblogic::Domain.new(name: 'otd_domain', admin_password: admin_password,
root_directory: '/oracle/app/otd/environments',
'admin_server.listen_port': 9090,
'admin_server.ssl.listen_port': 9190, restricted_jrf: true, templates: MintPress::OracleTrafficDirector::TemplateSet.new )
Note the ‘restricted_jrf: true’ setting, which causes OTD to not require access to a database server - most installations will desire this setup!
To create a simple configuration, the following ruby code would suffice:
# Create a configuration
domain.configuration['demo_otd'] = MintPress::OracleTrafficDirector::Configuration.new(listener_port: 19001, server_name: 'demo-otd.mintpress.io')
# Add an instance
domain.configuration['demo_otd'].add_instance('demootd-n1.mintpress.io')
# Add a server pool
domain.configuration['jj_otd'].add_origin_server_pool(name: 'obpsoa-ssl-origin-server-pool',
type: 'http',
'ssl.enabled': false,
'ssl.tls11': false,
'health_check.request_uri': '/soa-infra/services/isSoaServerReady',
'health_check.request_method': 'GET',
'health_check.interval': 4,
'health_check.timeout': 5,
'health_check.respond_code_match': 200)
# Add an origin server
domain.configuration['jj_otd'].origin_server_pool['obpsoa-ssl-origin-server-pool'].add_origin_server(name: 'obpsoa.demo.mintpress.io:8002')
# Add a virtual server
domain.configuration['jj_otd'].origin_server_pool['obpsoa-ssl-origin-server-pool'].add_virtual_server(name: 'obpsoa-e-virtual-server',
host: 'obpsoa.mintpress.io',
'access_log.format': '%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] \"%Req->reqpb.clf-request%\" %Req->srvhdrs.clf-status% %Req->srvhdrs.content-length% %Req->vars.ecid% %Req->vars.origin-server% %duration%',
'perf_dump.uri': '/perf'
)
# Run the create action to ensure the configuration!
domain.configuration['jj_otd'].create
To restart an instance , you can call #start and #stop on it (they are children of the Oracle Weblogic System Component class, and all methods apply to them as well)
domain.configuration['demo_otd'].add_instance('demootd-n1.mintpress.io').start
domain.configuration['demo_otd'].add_instance('demootd-n1.mintpress.io').stop
For examples of use with other integrations, such as Chef and Ansible, please see the included demo recipes.
About managing origin servers
It is now possible to drain the servers via the SDK, using the start_draining and disable_when_drained actions. The reason these are seperated, is to enable draining multiple origin servers at once - a fairly common situation with complex applications. If you have a JSON file for your OTD that you built with your runtime, the code can be as simple as this:
require 'mintpress-oldworld-integration'
require 'oracle-otd'
# We want to drain any origin servers pointing to this
shorthost = "origintest"
# Load the model in from a json file
json_data = JSON.parse("otd.json")
model = EnvironMintModelUtils::MintModel.new(json_data)
otd_model = MintPress::Migration::ModelProcessor.new(model, installations_fatal: false, templates_fatal: false, host_options: , generic_component_okay: true)
# Grab the OTD domain
otd_domain = otd_model.domains.values[0]
# Iterate through the configurations
draining_pools = []
otd_domain.otd_configuration.values.each do |otd_configuration|
# Iterate through the origin server pools
otd_configuration.origin_server_pool.values.each do |otd_origin_pool|
# Lets iterate the OTD origin servers for any which match our shorthost
otd_origin_pool.origin_server.values.each do |otd_origin_server|
if otd_origin_server.host.start_with?(short_host)
# Start this pool draining
otd_origin_server.start_draining
# Record that we did it
draining_pools << otd_origin_server
# Optionally, set a property to limit how long we drain for
otd_origin_server.drain_timeout = 300
end
end
end
end
# If you want to handle this manually, this is exactly what the below static function does
=begin
# Now disable pools as they become drained
draining_pools.each do |pool|
draining_pools.disabled_when_drained
end
# Lets enable them again straight away...
draining_pools.each do |pool|
draining_pools.enable
end
=end
MintPress::OracleTrafficDirector::OriginServer.drain_server_set(draining_pools, timeout: 300)
Alternatly, you can harvest your configuration from the OTD instead via code that looks like this:
# Grab a host of our admin servers
admin_host = Host.new(name: admin_host_name, connect_user: 'mintpress', final_user: 'oracle', keys: '~/.ssh/id_rsa')
# Grab a domain
domain = Domain.new(name: domain_name, host: admin_host, admin_user: 'weblogic', admin_password: 'welcome1', 'admin_server.listen_port': 7001)
# Inform the system that this is OTD - if you already know your configuration name, you can instead do:
#
# configuration = MintPress::OracleTrafficDirector::Configuration.new(name: config_name, domain: domain)
#
domain.templates << MintPress::OracleTrafficDirector::Template.new
# Harvest the domain on access
domain.harvest_on_access = true
# Iterate through configurations - accessing configuration.keys addresses a ruby
# quirk where `each` will sometimes happen without passing through to the harvest
# path
puts "Iterating: #{domain.templates[-1].configuration.keys.inspect}"
domain.otd_configuration.values.each do |otd_config|
puts "Working on #{otd_config.name}"
end
If you already know the configurations you’re going to work on, and just need to worry about the origin servers, the code can be simpler:
config_name = 'otd_test'
# Grab a host of our admin servers
admin_host = Host.new(name: admin_host_name, connect_user: 'mintpress', final_user: 'oracle', keys: '~/.ssh/id_rsa')
# Grab a domain
domain = Domain.new(name: domain_name, host: admin_host, admin_user: 'weblogic', admin_password: 'welcome1', 'admin_server.listen_port': 7001)
configuration = MintPress::OracleTrafficDirector::Configuration.new(name: config_name, domain: domain)
# Iterate through the configuration object as in above examples!
Handling multiple processes
Generally, it’s recommended to do all of your OTD manipulation from a single process, but if you have a requirement to run multiple processes in parallel, it’s important to synchronize them so they don’t grab the edit lock at the same time. For a small number, you can set the environment variable MINTPRESS_EXCLUSIVE_WLST to true, which will cause the WLST session to wait for any edit lock to be cleared before asserting its own.
This has a major disadvantage, however, that each of your processes are still running a WLST session. Each of these will take around 512mb of ram on your OTD server, which is obviously will cause problems if you run, say, 20 of them. A better approach is to lock the WLST with code like this:
# This function tests we have one and only one WLST open at any time
def ensure_wlst_available(origin)
while origin.domain.adminserver_transport.execute("ps axww | grep -v grep | grep weblogic.WLST | wc -l").stdout.to_i > 0
puts "Waiting for WLST session on OTD domain"
# In case a harvest opened a session, ensure we close it.
origin.domain.wlst_session.close if domain.has_open_wlst?
sleep 10*rand()
end
end
# This function waits for a WLST session to be drained
def wait_for_drain_only(origin, timeout: nil)
while true
# Get our connection count using a locked wlst, which we then immediatly close
ensure_wlst_available(origin)
# You can use different metrics here!
connection_count = origin.get_metric(['countActiveConnections', 'countActiveStickyConnections', 'requests5MinuteAverage']).inject(0, :+)
# If the connection count is 0, _or_ we have exceeded timeout, then we return
if connection_count == 0 or (self.drain_timeout.nil? or TIme.now.to_i - @drain_start_time <= self.drain_timeout)
origin_server.disable
origin.domain.wlst_session.close
return
end
sleep 30*rand()
origin.domain.wlst_session.close
end
end
# This function disables a set of origin servers
def disable_servers(origin_server_list)
# Start all of our servers draining
ensure_wlst_available(origin_server)
origin_server_list.each do |origin_server|
origin_server.start_draining
end
# Close the session so others can use it
origin_server.domain.wlst_session.close
# Now wait for drains to complete
origin_server_list.each do |origin_server|
wait_for_drain_only(origin_server, timeout: 300)
end
end
# This function enables the origin servers
def enable_servers(origin_server_list)
ensure_wlst_available(origin_server)
origin_server_list.each do |origin_server|
origin_server.enable
end
# Close the session so others can use it
origin_server.domain.wlst_session.close
end
Threaded Failover Start
Right now, a performance issue can be waiting for sudo authentication for failover - this happens in serial today, due to a race condition experienced on a client site, hence while we believe this issue is resolved, it is disabled by default out of an abundance of caution. If you would like to enable this parallel functionality, you can set the environment variable OTD_THREADED_FAILOVER to true, or the ruby global variable $otd_threaded_failover = true.
License & Authors
-
Author:: LimePoint (support@limepoint.com)
# MintPress® - Automation and Configuration Management Platform
# Copyright © 2018 LimePoint. All rights reserved.
#
# This program and its contents are confidential and owned by LimePoint.
# Only licenced users are permitted to access and use of this file.
# This program (or any part of it) may not be disclosed, copied or used
# except as expressly permitted in LimePoint’s End User Licence Agreement.
#
# LimePoint® and MintPress® are Registered Trademarks of LimePoint
# For more information contact LimePoint at http://www.limepoint.com