How are you all monitoring or measuring how much usage your OOD portals are getting? We can tell how many jobs are submitted through our portal but not how many users are logging in and transferring files, using the shell, or just monitoring their jobs.
If this is already asked and answered, I apologize! I thought I saw it somewhere but couldn’t find it in discourse or old mailing list emails.
Thanks!
Dori
UB CCR
Hi Dori:
Check
https://discourse.openondemand.org/t/ganglia-metrics/105
and
https://github.com/OSC/ondemand-metrics/archive/master.zip
That’s using ganglia to monitor usage, but the plugin could probably be munged to deal with other tools since it’s a Python script.
Cheers,
Ric
Dori,
If you create a Google Anallytics property and have the id you can add something like this:
analytics:
url: http://www.google-analytics.com/collect
id: UA-12345678-9
to the /etc/ood/config/ood_portal.yml config file and regenerate the Apache config, which in turn simply produces these directives at the bottom of <Location "/pun">:
SetEnv OOD_ANALYTICS_TRACKING_URL "http://www.google-analytics.com/collect"
SetEnv OOD_ANALYTICS_TRACKING_ID "UA-12345678-9"
LuaHookLog analytics.lua analytics_handler
</Location>
The result is a bunch of information about each request is sent to Google Analytics. Now I think in order for this to work you need to setup some extra custom fields in the Analytics property. I can hunt down the documentation for this if you are interested. The corresponding lua code that sends this information is here.
That said, this would only work for future requests. All of the same information that is optionally sent to Google Analytics is now also logged to /var/log/httpd24/yourondemand.domain.edu_error_ssl.log and this can be consumed in Splunk or another analytics tool and you can do searches. Here is an example log from a request to the AweSim dashboard (an OnDemand instance at OSC):
[Wed Mar 13 13:42:18.110433 2019] [lua:info] [pid 13512] [client 131.187.116.31:20665] req_protocol="HTTP/1.1" req_handler="proxy-server" req_method="GET" req_accept="text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" req_user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:65.0) Gecko/20100101 Firefox/65.0" res_content_length="7216" req_content_type="" res_content_encoding="gzip" req_status="200" req_origin="" time_user_map="63.699" local_user="efranz" req_referer="https://idp-test.osc.edu/auth/realms/awesim/protocol/openid-connect/auth" res_content_language="" req_port="443" log_time="2019-03-13T17:42:18.110304Z" req_server_name="apps-test.awesim.org" log_hook="ood" req_accept_charset="" req_hostname="apps-test.awesim.org" res_content_location="" res_content_disp="" req_is_websocket="false" remote_user="efranz" res_location="" req_user_ip="131.187.116.31" req_is_https="true" req_filename="proxy:http://localhost/pun/sys/dashboard" req_uri="/pun/sys/dashboard" time_proxy="4730.865" res_content_type="text/html; charset=utf-8" req_accept_language="en-us,en;q=0.5" req_cache_control="" req_accept_encoding="gzip, deflate, br", referer: https://idp-test.osc.edu/auth/realms/awesim/protocol/openid-connect/auth?response_type=code&scope=openid&client_id=apps-test.awesim.org&state=wmBY8FHRSaBRv8iE2_lfgO1O16I&redirect_uri=https%3A%2F%2Fapps-test.awesim.org%2Foidc&nonce=Lgqu4ayoX0i4vlh1QoG5IOMyTL4kk54jmFVhLr-HTFg
If you do searches, you will want to play games with the req_uri value since you might only be interested in knowing a request to the Dashboard app (req_uri="/pun/sys/dashboard") occurred by a user on a particular day, and omit the extra requests for things like images and css etc. (where the req_uri’s all are prefixed with /pun/sys/dashboard).
I’m happy to share example Splunk search strings using this data if it would be helpful.
Thank you very much, Ric!
Thanks Eric. I’ll try this and get back to you if I have any questions. We’re not using Splunk here.