Authentic8 API Log Extraction Reference Scripts

Summary

Example code provided below is for reference only. Authentic8 strongly encourages you to develop your own scripts using the language and coding practices best suited for your organization.


Logs are stored in the Authentic8 databases and can only be accessed programmatically via the Authentic8 API.

 

An Authentication Token is a security credential required for programmatic access to the Authentic8 API. See the Authentic8 API Reference Guide for more information. Authentication tokens are issued by the Authentic8 Support team on a per Admin User request basis to access Authentic8 API.  Contact Support to get your token.

 

Log Extraction Reference (unencrypted logs)

For samples scripts below to extract clear test logs, the parameters are:

  • -o 'org name'

  • -t 'auth token file'

  • -i <start_sequence_id>

  • -I <end_sequence_id>

  • -d <start date>

  • -D <end date>

 

One of -i or -d is used to specify the starting point of the log extraction. Optionally, -I or -D can be used to specify the end point of the log extraction.

 

Supported Log Types are: URL, DOWNLOAD, UPLOAD, SESSION, AUTH (authentication logs for Silo session ONLY), ADMIN_AUDIT, LOCATION CHANGE, BLOCKED URL, TRANSLATION, & A8SS

 

The result is a JSON object, which has three keys at the top level

  • is_more: a boolean which is true if more log files are available

  • next_seq: a sequence number to use with -i for the next set of log files

  • logs: a list of JSON objects representing the log lines.

 

Sample cURL command for extracting clear text logs:

 

curl -kv -H "content-type: application/json" -X POST -d '[{"command": "setauth", "data": "<AuthToken>"},{"command": "extractlog", "start_seq": 0, "org": "<OrgName>", "type": "URL"}]' https://extapi.authentic8.com/api/

 

Note: On Microsoft Windows systems, you’ll need to install additional open source software like Cygwin to use the cURL command-line tool as shown in this example.

 

Sample Python script for extracting clear text logs:

 

#!/usr/bin/env python


from getopt import getopt # Old school to support Python < 2.7

import json

import sys

import urllib2

 

def usage_abort( extra='' ):

   sys.exit( extra + '''

Usage: log_extract -o <org name> -t <auth token file>

                  [ -i <start id> | -d <start date> ]

                  [ -I <end id> | -D <end date> ]

Example:

log_extract -o "Customer org" -t auth.txt -d "2014-04-01 14:00:00"''' )

 

opt_array, args = getopt( sys.argv[1:], 'd:i:D:I:l:o:t:' )

if args != []:

   usage_abort( ' '.join( args ) + ' would be ignored' )

opts = dict( opt_array )

ea_host = 'extapi.authentic8.com'

if '-o' in opts:

   org = opts['-o']

else:

   usage_abort( 'Missing org' )

 

cmd = {

   'command': 'extractlog',

   'org': org,

   'type': 'URL' }

if '-i' in opts:

   cmd['start_seq'] = int( opts['-i'] )

if '-d' in opts:

   cmd['start_time'] = opts['-d']

if '-I' in opts:

   cmd['end_seq'] = int( opts['-I'] )

if '-D' in opts:

   cmd['end_time'] = opts['-D']

if not ( 'start_seq' in cmd or 'start_time' in cmd ):

   cmd['start_seq'] = 0

if '-l' in opts:

   cmd['limit'] = int( opts['-l'] )

t = open( opts['-t'], 'rb' )

auth_cmd = {

   'command': 'setauth',

   'data': t.read().strip()}

t.close()

 

req = urllib2.Request( 'https://' + ea_host + '/api/',

                      json.dumps( [ auth_cmd, cmd ] ),

                      { 'Content-Type': 'application/json' } )

reader = urllib2.urlopen( req )

res = json.loads( reader.read() )

reader.close()

assert len( res ) == 2

if 'result' in res[1]:

   print json.dumps( res[1]['result'], indent=2, ensure_ascii=False )

else:

   print 'Failure'

   import pprint

   pprint.pprint( res[1]['error'] )

 

Log Extraction Reference (encrypted logs)

Extracting and decrypting encrypted logs requires configuring a log encryption policy within Authentic8 and uses the free SECCURE ECC public key encryption toolset, which is licensed under the GNU Lesser General Public License v3 (LGPL).  

 

Source packages and documentations can be found at: http://point-at-infinity.org/seccure/

 

See the installation instructions for the Python SECCURE library at:

https://github.com/bwesterb/py-seccure

 

General Info on the log encryption feature can be found here

 

For samples scripts below to extract encrypted logs, the parameters are:

  • -o 'org name'

  • -t 'auth token file'

  • -p 'passphrase file'

  • -i <start_sequence_id>

  • -I <end_sequence_id>

  • -d <start date>

  • -D <end date>

 

One of -i or -d is used to specify the starting point of the log extraction.  Optionally, -I or -D can be used to specify the end point of the log extraction.

 

Supported Log Type is ENC for encrypted logs.

 

After extraction and decryption, the regular Authentic8 log types (AUTH, SESSION, et al.) may be identified, as well as additional POST DATA and  COOKIES log types. POST DATA and COOKIES log types are only available when using log encryption.

 

The result is a JSON object, which has three keys at the top level

  • is_more: a boolean which is true if more log files are available

  • next_seq: a number to use with -i for the next set of log files

  • logs: a list of JSON objects representing the log lines.

 

Sample cURL command for extracting encrypted logs:

 

curl -kv -H "content-type: application/json" -X POST -d '[{"command": "setauth", "data": "<AuthToken>"},{"command": "extractlog", "start_seq": 0, "org": "<OrgName>", "type": "ENC"}]' https://extapi.authentic8.com/api/

 

Note: On Microsoft Windows systems, Cygwin can be installed to use the cURL command-line tool.

 

Sample Python script for extracting encrypted logs:

 

#!/usr/bin/env python


import base64

from getopt import getopt # Old school to support Python < 2.7

import json

import seccure

import sys

import urllib2

 

def usage_abort( extra='' ):

   sys.exit( extra + '''

Usage: enc_log_extract -o <org name> -t <auth token file> -p <passphrase file>

                  [ -i <start id> | -d <start date> ]

                  [ -I <end id> | -D <end date> ]

Example:

enc_log_extract -o "Customer org" -t auth.txt -p pass.txt -d "2014-04-01 14:00:00"''' )

 

opt_array, args = getopt( sys.argv[1:], 'd:i:D:I:l:o:p:t:' )

if args != []:

   usage_abort( ' '.join( args ) + ' would be ignored' )

opts = dict( opt_array )

ea_host = 'extapi.authentic8.com'

if '-o' in opts:

   org = opts['-o']

else:

   usage_abort( 'Missing org' )

if '-p' in opts:

   pass_file = opts['-p']

else:

   usage_abort( 'Missing passphrase file' )

 

cmd = {

   'command': 'extractlog',

   'org': org,

   'type': 'ENC' }

if '-i' in opts:

   cmd['start_seq'] = int( opts['-i'] )

if '-d' in opts:

   cmd['start_time'] = opts['-d']

if '-I' in opts:

   cmd['end_seq'] = int( opts['-I'] )

if '-D' in opts:

   cmd['end_time'] = opts['-D']

if not ( 'start_seq' in cmd or 'start_time' in cmd ):

   cmd['start_seq'] = 0

if '-l' in opts:

   cmd['limit'] = int( opts['-l'] )

t = open( opts['-t'], 'rb' )

auth_cmd = {

   'command': 'setauth',

   'data': t.read().strip()}

t.close()

p = open( pass_file )

passphrase = p.read().rstrip()

p.close()

 

req = urllib2.Request( 'https://' + ea_host + '/api/',

                      json.dumps( [ auth_cmd, cmd ] ),

                      { 'Content-Type': 'application/json' } )

reader = urllib2.urlopen( req )

res = json.loads( reader.read() )

reader.close()

assert len( res ) == 2

if 'result' in res[1]:

   for l in res[1]['result']['logs']:

        l['clear'] = json.loads(

            seccure.decrypt( base64.b64decode( l['enc'] ),

                             passphrase, curve='secp256r1/nistp256' ) )

   print json.dumps( res[1]['result'], indent=2, ensure_ascii=False )

else:

   print 'Failure'

   import pprint

   pprint.pprint( res[1]['error'] )