SOC Pt 1: Elastic, Tines, and Jira
A few months ago, I was reading Coinbase's blog series about how they scaled their detection and response operations, and aside from the SecurityBot, I was really interested in the platform they built to centralize their investigations. I believe I was looking at one of their job descriptions when I noticed that they had Tines mentioned. I had never heard of it until then. I went to the website and it seemed really cool. Workflow automation was something I was already heavily considering for a project outside of security, and I wanted to find an excuse to sign up and start a project.
Fast forward to a few weeks ago, I figured out what that project would be. I want to create an at-home SOC using Elastic, Tines, and Jira. I want alerts to go from Elastic to Tines, use Tines to automatically enrich the alert, and then send that information to a queue like Jira. Sounds pretty simple and doable. I figured the best way to start would be to just hop right in.
Setting up Elastic Security
I started a free trial of Elastic using the serverless option on AWS. It was really seamless. After some quick naming and configuration, I installed all the preconfigured detection rules, which looking back I probably shouldn’t have done right away.
I used Elastic's Fleet to create an agent policy, then installed it using the script on one of my devices. Once again, it was seamless, and I was collecting logs.
I spent more time looking around and configuring things here and there, mostly in the agent policy and the detection rules. I have worked extensively in Elastic as an analyst but not as an "engineer".
After I got the basics setup, I pivoted to setting up Jira.
Setting up Jira Service Management and Tines
Next, I created a Jira account and chose the Jira Service Management option. I configured the basics, created an API key, and connected it to Tines.
In Elastic, I configured the Tines connector, which was also quick and very easy. I then bulk-updated all the rules to send alerts to my Tines story, and changed the setting from "Summary of alerts" to "For each alert." That was the only change I needed to make.
Testing the Jira integration in Tines wasn’t easy. I ran into a bunch of issues and kept getting this error:
Failed with 401 status code
{"errorMessages":["You do not have permission to create issues in this project."],"errors":{}}
I read a lot of documentation, tried switching actions to using the Jira SM action template in Tines, but I just kept going in circles. I eventually found an Elastic story template called “Normalize alerts from multiple sources using ChatGPT” and that immensely helped me understand how credentials and parsing actually works in Tines.
I was able to reference the actions and flow in that template and realized I wasn’t associating the credentials with the actions at all, which is why I kept getting permission errors. I also learned how resources work in Tines, and how useful they are.
My original goal was to enrich alerts with VirusTotal, AbuseIPDB, etc., but I ended up pivoting to ChatGPT integration for the time being. I set it up so that alerts would go to Tines, get sent to ChatGPT to summarize and pull relevant information, then have the JSON parsed and sent to Jira.
Looking back, I probably could’ve just used the Jira connector in Elastic to send alerts straight to Jira. But that kind of defeats the purpose of the project, which was to explore Tines.
Fine-Tuning
Once I got the automation flow working, it was cleanup time.
I removed all the default rules in Elastic, re-enabled only the ones relevant to my environment, re-added the alert actions, and tested alerts multiple times to make sure everything was flowing smoothly.
At this point, I had installed the agent on another computer and noticed one of my endpoints was marked as unhealthy. I had enabled integrations like System and Windows, but forgot to install Sysmon. I fixed that quickly and got everything healthy again.
From there I just kept testing over and over with different rules, fixing small things, tweaking the ChatGPT prompt, editing the Jira issue format, etc, just dialing it in.
It was very rewarding to see alerts funnel into Jira almost immediately after they were detected in Elastic. Some of the alerts were even insightful, such as one that let me know I still had Splashtop installed on one of my hosts.
What's Next
My free trial of Elastic just ended, but I’ve signed up and will continue working on this project. I’ve got a few new goals I plan to cover in a future update:
- Switch from summarizing alerts with OpenAI to directly parsing fields from the raw alert JSON
- Enrich alerts using threat intel sources like VirusTotal, AbuseIPDB, etc., tools that would save me time in a real SOC
- (Maybe) still use OpenAI to suggest analyst actions or next steps
- Replace my current AV with CrowdStrike Falcon across all hosts
- Ingest Falcon alerts into Elastic
- Deploy Elastic Agent to all hosts
- Enjoy a home environment that operates like a real SOC
If I were doing this in a professional setting, I’d build it out to support multiple clients, set SLA timers, and spend more time optimizing the Jira environment for better tracking and response. But good thing this is not a professional setting.
This has definitely been one of my favorite projects so far, and I’m looking forward to continuing to improve it. I’ve wanted a SIEM in my home environment for a long time, both for hands-on practice and added visibility, and Elastic makes it so easy to get started that there’s really no excuse not to.
If I expand my log sources to include things like firewall logs, etc., it’ll be pretty close to having a mock production environment. I could even take it a step further by introducing a VM to simulate actual threats and test how well my detections, automations, and response workflows hold up in more realistic scenarios.
I really want to do that actually, but I am going to focus on the tasks that I have listed above before getting too ahead of myself.