Student: Shubham Pandey (@sp35)
Mentor: Eshaan Bansal and Matteo Lodi
Organization: The Honeynet Project
Project: Intel Owl
Tag: Information Security
Intel Owl is an Open Source Intelligence or OSINT solution to get threat intelligence data about a specific file, an IP, or a domain from a single API at scale. It integrates a number of analyzers available online and is for everyone who needs a single point to query for info about a specific file or observable.
As cited in my original proposal’s overview:
I propose a new component of the project – a connectors manager which would help IntelOwl connect with any external tool for enhancing the data it generates or performing automated analysis on the data it generates, or facilitate the integration with other threat intelligence platforms. Along with this, I would also add the integrations for MISP and OpenCTI platforms which the community has been eagerly waiting for.
So the main objective was to develop a connectors manager which would help IntelOwl connect with any SOAR/SIEM platform, particularly for threat-intel sharing purposes.
List of pull requests merged before GSoC’s coding period:
A common pattern observed in analyzers’ python modules was the validation of secrets configured via a JSON configuration file. Also, a user while requesting an analysis, couldn’t know if an analyzer was configured properly or not. This was something that would affect the new component – connectors as well.
I and Sarthak (who worked on the analyzers part of this issue) used DRF’s serializers to validate the configuration file initially and cached it for the rest of the application’s lifecycle while invalidating the cache if the file was changed. We used the design pattern `Abstract Factory` and created abstract classes to extend this for both analyzers and connectors (#499, #518).
Finally, it was time to work on some integrations using connectors. Integrations were added for threat-intel sharing platforms – MISP (#528), OpenCTI (#602), YETI (#631). It also made sense to leverage the opportunity to add analyzers that would search for observables/threat-intel reports available on these platforms – OpenCTI (#603), YETI (#632).
I proposed that connectors would have a `kill` feature to stop any ongoing/pending connector run and `retry` to restart any failed/killed connector run. Another action health-check was considered to check if the associated instances (docker containers and external platforms) were up or not, anytime, instead of finding out later in a failed analysis. Once again abstract classes were used to extend these to both analyzers and connectors.
All the corresponding user interface changes were committed to the repo IntelOwl-ng (Angular app).
All the corresponding changes to the python client were committed to the repo Pyintelowl (python client).
Added CLI commands and library methods for fetching connectors’ configuration (#112), plugin actions kill/retry (#113), and health-check (#114).
I’m very glad to have worked on this amazing project and would continue to do so even after GSoC. In the future, we shall be working on these features:
I would like to thank The Honeynet Project and Google Summer of Code for providing me with this opportunity and especially my mentors Eshaan Bansal and Matteo Lodi for being kind and helpful mentors to me along this amazing journey.