log-analytics & import_logs.py, hosts parameter failing · Issue #6919 · matomo-org/matomo · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log-analytics & import_logs.py, hosts parameter failing #6919

Closed
mpiscaer opened this issue Jan 5, 2015 · 2 comments
Closed

log-analytics & import_logs.py, hosts parameter failing #6919

mpiscaer opened this issue Jan 5, 2015 · 2 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone

Comments

@mpiscaer
Copy link

mpiscaer commented Jan 5, 2015

Hi,

Like I sad on the forum: http://forum.piwik.org/read.php?2,123133,123166#msg-123166.

I try to import some logs from the nginx webserver into piwik 2.9.1.

I have followed the howto from https://github.com/piwik/piwik/tree/master/misc/log-analytics#setup-nginx-logs.

This setup generates a nice access log file that looks like:

{"ip": "192.168.1.10","host": "www.test.nl","path": "/","status": "200","referrer": "https://www.test.nl/","user_agent": "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36","length": 915,"generation_time_milli": 0.001,"date": "2015-01-02T13:20:31+01:00"}

When I try to import this access log with:
$ python /var/www/misc/log-analytics/import_logs.py --url=http://webanalyse.domain.local/ --add-sites-new-hosts --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --log-format-name=nginx_json --config /var/www/config/config.ini.php access.log

I get the flowing error:

0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log access.log...
Traceback (most recent call last):
File "/var/www/misc/log-analytics/import_logs.py", line 1750, in
main()
File "/var/www/misc/log-analytics/import_logs.py", line 1717, in main
parser.parse(filename)
File "/var/www/misc/log-analytics/import_logs.py", line 1576, in parse
resolver.check_format(format)
File "/var/www/misc/log-analytics/import_logs.py", line 1119, in check_format
elif 'host' not in format.regex.groupindex and not config.options.log_hostname:
AttributeError: 'NoneType' object has no attribute 'groupindex'

As I understands is correctly is it missing the host parameter in the access log but as you can see it is there. What goes wrong?

Kinds regards,

Michiel Piscaer

@mpiscaer
Copy link
Author

mpiscaer commented Jan 5, 2015

After upgrading the installation to 2.10.0 I still get the flowing message:

2015-01-05 14:05:54,763: [DEBUG] Accepted hostnames: all
2015-01-05 14:05:54,764: [DEBUG] Piwik URL is: http://webanalyse.domain.local/
2015-01-05 14:05:54,764: [DEBUG] Authentication token token_auth is: xxxxxxxx
2015-01-05 14:05:54,764: [DEBUG] Resolver: dynamic
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
2015-01-05 14:05:54,766: [DEBUG] Launched recorder
2015-01-05 14:05:54,766: [DEBUG] Launched recorder
2015-01-05 14:05:54,767: [DEBUG] Launched recorder
2015-01-05 14:05:54,767: [DEBUG] Launched recorder
Parsing log access.log...
Traceback (most recent call last):
File "/var/www/misc/log-analytics/import_logs.py", line 1900, in
main()
File "/var/www/misc/log-analytics/import_logs.py", line 1871, in main
parser.parse(filename)
File "/var/www/misc/log-analytics/import_logs.py", line 1689, in parse
resolver.check_format(format)
File "/var/www/misc/log-analytics/import_logs.py", line 1234, in check_format
elif 'host' not in format.regex.groupindex and not config.options.log_hostname:
AttributeError: 'NoneType' object has no attribute 'groupindex'

@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Jan 6, 2015
@mattab mattab added this to the Short term milestone Jan 6, 2015
@mattab
Copy link
Member

mattab commented Jan 6, 2015

Hi @mpiscaer thanks for the report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

2 participants