VAST Threat Bus now runs up to 500 queries in parallel by default, and waits for an hour before aborting a query. This makes use of VAST v2.0's new query scheduling mechanism.

Thanks to a contribution from Sascha Steinbiss (@satta), Threat Bus only reports failure when transforming a sighting context if the return code of the transforming program indicates failure.

A small peek behind the curtain: We’re building the next generation of Threat Bus as part of VAST. We will continue to develop and maintain Threat Bus and its apps for the time being.

Dear users, we are happy to announce Threat Bus 2021.12.16! 🎉

Threat Bus now correctly post-processes sightings generated by the VAST Matcher plugin when using probabilistic filters. Due to the nature of probabilistic filters, generated sightings in STIX-2 format cannot be correlated with the indicators they originated from, as the indicator ID is no longer available. The generated STIX-2 sighting instead contains a fixed indicator ID of note--00000000-0000-4000-8000-000000000000 that represents a valid UUID unlikely to be used in practice by other tools.

Read the full CHANGELOG here.

This release of Threat Bus fixes a bug in the support for low-priority queries that snuck into Threat Bus 2021.11.18.

This release of Threat Bus adds the ability to run low-priority queries against VAST.

Threat Bus 2021.09.30 is purely a maintenance release, and contains no user-facing changes.

This patch release fixes incorrect dependency versions of the stix2 package in threatbus, and an incorrect dependency version on the threatbus package in vast-threatbus. These were missed in the original release due to a mismatch between the requirements.txt and setup.py files.

This month's release most prominently features some restructuring in the Threat Bus packages. This includes a breaking change for users of the zmq-app plugin, see the paragraph below.

For VAST Threat Bus, support for live matching in VAST has been restored. It had been disabled after a refactoring on the VAST side.

To work around some UX issues with dashes in nested configuration options, we decided to rename the Threat Bus ZMQ App to Threat Bus ZMQ. Users of this plugin will need to update their config files to move the Threat Bus ZMQ configuration from plugins.zmq-apptoplugins.zmq`.

To combine similar breaking changes in the same release, we also renamed PyVAST Threat Bus to VAST Threat Bus in order to get a more consistent naming scheme.

Changelog Highlights

• 🎁 Live matching with VAST works again! #156

• 🐞 Fixed config validation for the 'apps.misp.api' setting. #161

• ⚡️ The threatbus-zmq-app package has been renamed to threatbus-zmq, to address some limitations in the configuration framework. #157

• ⚡️ We renamed PyVAST Threat Bus to VAST Threat Bus for clarity. The PyPI package name and the binary name change from pyvast-threatbus to vast-threatbus accordingly. #159

This release of Threat Bus comes with a complete overhaul of the config system: it is now powered by Dynaconf, which brings along a bag of goodies:

• All config values can now be overwritten using environment variables
• Support for config file validation
• Secrets can be read from a separate secrets file or the environment

Additionally, most config values have been assigned default values, making it possible to start Threat Bus with a far more minimal configuration file than before.

Another important change concerns the Threat Bus Apps: The content and format of the threatbus-zmq-app plugin's subscription success response has changed. Prior to this change, the plugin used to respond with an endpoint in the host:port format, which might contain a wrong hostname (e.g., 0.0.0.0 instead of a publicly reachable topic). From now on, the plugin returns only the ports for pub and sub communication and leaves it to the subscribing app to connect with the right host/IP.

We also improved the metrics subsystem of the VAST Threat Bus app: The metric for indicator query time now only reflects the actual time spent querying VAST and no longer regards unstarted VAST queries. Metrics sent by the app now use the fully qualified domain instead of just the hostname to identify the sending machine. And we fixed the serialization format to ensure all fields are separated by commas, so that the output conforms to the Influx line protocol spec

Changelog Highlights

You can find the full Changelog here.

• ⚠️ Threat Bus now uses Dynaconf for configuration management. Configuration via a config file works exactly as it has worked before. Users can provide a path to the config file using the -c option. Threat Bus now considers files named config.yaml and config.yml as default configs if located in the same directory. Additionally, Threat Bus now supports configuration via environment variables and .dotenv. Environment variables need to be prefixed with THREATBUS_ to be respected and always take precedence over values in config files. #133

• 🐞 Threatbus now only attempts to load plugins that are explicitly listed in the config file. #150

• 🎁 Many configuration options for threatbus and pyvast-threatbus now have default values. See the example configs for a detailed list. #150

We’re happy to announce our release 2021.06.24 of Threat Bus.

One important update concerns our community. We finally consolidated our Gitter chats into a Slack Community. Join us in the #threatbus channel for vibrant discussions.

Suricata Integration

A new month, a new Threat Bus app! We have implemented initial support to connect Suricata to Threat Bus. The main use case for the popular network monitor and IDS is rule-based alerting. Luckily, Suricata rules are valid pattern types in STIX-2.1 indicators and hence Threat Bus can already transport them.

The new Suricata app works similar to pyvast-threatbus and stix-shifter-threatbus in that it communicates via ZeroMQ. It subscribes to the STIX-2 indicator stream in Threat Bus and picks up all indicator domain objects where the STIX-2 pattern type equals suricata. The Suricata rules in those indicators are then forwarded to Suricata using a configurable rules file, which the app periodically reloads via UNIX domain sockets using suricatasc.

Suricata only supports hot reloading of rules through a file, which is the reason whysuricata-threatbus maintains its own rules file. It would be nice if there was a path to directly push rules into Suricata, without the need to go through files. There are also other types of security content that users can configure in Suricata. For example, IP reputation lists (likewise file-based) and Datasets. Our Suricata app will leverage these structures in the future and synchronize them with generic STIX indicators. Especially datasets hold promise as generic carrier for tactical TI. If you are interested in the matter, please also read this post in the Suricata forum and check the linked issues for updates.

With suricata-threatbus, Suricata users can now finally benefit from the rich integration ecosystem Threat Bus has to offer. For example, with a STIX-based threat intelligence platform like OpenCTI, you can now also manage Suricata rules along with your security content, and, thanks to our OpenCTI Threat Bus integration, updates to those Suricata rules are immediately published on the bus, which in turn live-updates all your Suricata instances. With our all-new Suricata app, users can now seamlessly integrate intelligence from OpenCTI or MISP with Suricata. Stay tuned for future updates and integrations!

Sightings Backchannel for STIX-Shifter

With last month’s release we have published stix-shifter-threatbus. The Threat Bus app leverages STIX-Shifter to transform STIX-2 indicators from Threat Bus into native queries for a huge set of commercial security tools and SIEMs. Now stix-shifter-threatbus just got a little better and is finally able to report back query results in the form of STIX-2 sightings. Sightings are forwarded to Threat Bus via ZeroMQ and subscribers receive them via their usual topic subscriptions on stix2/sighting.

Users can now fully integrate their Splunk, IBM QRadar, ElasticSearch SIEM, and many more tools with Threat Bus. For example, you can easily maintain your intelligence with OpenCTI, forward updates to your SIEM in near-real time and get query results (sightings) reported back in, again, near real time. We’re excited to fuel integration of awesome tools with Threat Bus!

Smaller Things

• We have dockerized pyvast-threatbus and stix-shifter-threatbus. Both projects are available on Dockerhub.
• pyvast-threatbus now collects metrics about received indicators that are about to be matched retrospectively against VAST. The new metric is called retro_match_backlog and allows users to determine if a backlog is building up.
• The Threat Bus Docker base image has moved to debian:bullseye for improved Zeek/Broker support.

Changelog Highlights

As always, you can find the full scoop in our various changelogs for Threat Bus and all Tenzir-maintained apps: pyvast-threatbus, stix-shifter-threatbus, and suricata-threatbus. Please also check out our OpenCTI connector over in the official OpenCTI repository.