YouTube Deep SummaryYouTube Deep Summary

Star Extract content that makes a tangible impact on your life

Video thumbnail

How To Setup ELK | Elastic Agents & Sysmon for Cybersecurity

John Hammond • 2023-05-10 • 14:35 minutes • YouTube

🤖 AI-Generated Summary:

Getting Started with ELK Stack for Security Monitoring: A Practical Walkthrough

Setting up a Security Information and Event Management (SIEM) solution can often feel overwhelming due to the many components involved—Elastic, Logstash, Kibana, and more. Recently, I embarked on a journey to demystify this process, using free and accessible resources from John Strand’s anti-phishing training and Black Hills Information Security’s pay-what-you-can courses. In this blog post, I’ll share my step-by-step experience setting up the ELK stack (Elasticsearch, Logstash, Kibana) for security monitoring, including installing agents and configuring Sysmon logs to capture meaningful Windows event data.


Why ELK Stack for Security?

The ELK Stack is a powerful open-source platform for searching, analyzing, and visualizing log data in real time. When combined with Sysmon (System Monitor), it becomes a valuable tool for defenders to track malicious activity, understand attacker tradecraft, and build detection capabilities using the MITRE ATT&CK framework.


Step 1: Access Free ELK Labs and Training

Before diving into the setup, I want to highlight the fantastic resources that helped me along the way:

  • John Strand’s Introductory Labs: Available freely on GitHub (search “strandJS intro labs”), these labs guide you through security tools and techniques.
  • Black Hills Information Security's Pay-What-You-Can Training: These courses cover active defense, cyber deception, and more. You can register for free tuition assistance if needed, making this education accessible to everyone.

These resources provide written walkthroughs and labs that are beginner-friendly and well-documented.


Step 2: Creating Your ELK Deployment in Elastic Cloud

To get started quickly without complex local installs:

  1. Visit the Elastic Cloud website and sign up for a free 14-day trial (no credit card required).
  2. Fill out a simple form with your email and create a password.
  3. Create a new deployment (I named mine “Security Deployment”) using the default settings.
  4. After deployment creation, note the root credentials provided (they appear only once, so save them securely!).

The Elastic Cloud gives you access to Kibana for visualization, Elasticsearch for data storage, and Fleet for managing agents—all hosted in the cloud.


Step 3: Adding Elastic Agents with Fleet

Elastic Agents collect data from your endpoints and send it to your ELK deployment.

  • Navigate to Fleet in Kibana under Security Management.
  • Create a new agent policy or use the default.
  • Click Add Agent, choose your host OS (e.g., Windows), and copy the enrollment command.
  • Run this command in an elevated PowerShell terminal on your Windows machine.

This process downloads, installs, and enrolls the Elastic Agent as a service on your host, enabling real-time data collection.


Step 4: Installing and Configuring Sysmon

Windows logs alone can be noisy and not always useful for security monitoring. Sysmon enhances Windows event logging by capturing detailed process creation, network connections, and more.

  • Download Sysmon from the official Sysinternals site.
  • Extract the zip file and install Sysmon with the command:

sysmon64.exe -i

  • Confirm that the Sysmon service is running by checking Windows services.

Step 5: Configuring Fleet to Collect Sysmon Logs

Back in Kibana:

  • Open your agent policy in Fleet.
  • Add the Windows integration.
  • Enable the Sysmon module within the integration.
  • Save and deploy the changes.

With this configuration, Elastic Agents will collect enhanced Sysmon logs and send them to your ELK deployment for analysis.


Step 6: Generating and Viewing Logs in Kibana

To verify data collection:

  • Perform some activities on the Windows host like opening applications (Calculator, WordPad), browsing, or running commands.
  • In Kibana, go to the Discover dashboard.
  • Select the logs data view and apply filters for windows.sysmon_operational logs.
  • Search for event ID 1 (process creation) to see detailed logs of process starts with command lines and timestamps.

This visibility allows you to track what processes are running on your endpoints and when, providing valuable information for threat detection.


Why This Matters

This hands-on experience with ELK Stack and Sysmon shows how defenders can build their own SIEM to monitor endpoint activity effectively. The ability to ingest, search, and visualize security logs is crucial for understanding attacks and improving organizational defenses.


Final Thoughts and Resources

If you’re interested in diving deeper into security monitoring and defense, I highly recommend:

This journey not only made ELK approachable but also provided tactical knowledge applicable in real-world security operations.

Thanks for reading! If you found this helpful, consider subscribing to stay updated with more practical infosec guides.


About the Author

A passionate information security enthusiast dedicated to making complex security tools accessible through hands-on tutorials and clear explanations. Follow for more cybersecurity insights and tutorials.


Happy hunting and stay secure!


📝 Transcript (415 entries):

I have always wanted to do this for a video but it's been a little bit daunting because configuring elk elastic log stash Cabana this whole structure and setup for a seam solution or Sim however you pronounce it can be a little bit a lot of moving pieces right so I'm excited I'm stoked I'm super happy to be able to do this with the help of John strand's courses his introductory Labs that are freely available all online just as a gentle reminder you can always be jumping into any of John strands and anti- siphon training and black hills information security in this awesome tribe of companies pay what you can training if you haven't seen it it's just literally courses education free training that you can choose the price tag for but if you take a look they do have some incredible courses coming up like their active defense and cyber deception course and tons and tons more there's things that you could learn all about making hackers earn their access and making them cry when you're wasting their time doing some great defense in depth and tons of great stuff from John strand well he's always putting out a lot of these pay what you can training if you haven't registered for these before you just cruise through it hey fill out whatever forms you need to but you get down to the price section look you can pay the minimum you can pay 50 you can pay 95 but if you want to bring this down even lower to make it more accessible for you if you just don't have the cash it is pay what you can so for tuition assistance you can click here and then you'll get a new form where all of those pricing options go away and you just register and you sign up and that's it you can make this course free accessible to you there are tons of other pay what you can courses and it's always worth just taking a look at what is antiphon training up to what is black kills information security up to and hey how can I jump into Wild West hacking Fest their conference anyway let's get into their publicly accessible and free introductory Labs that are part of these pay what you can courses you can find them online just on GitHub strand JS intro labs and in the past couple of videos we set up a virtual machine where we've gotten a chance to play with a lot of these Labs but there are so many that you can just cruise through so in this video I want to get into elk elastic log stash Cabana and this is a three-part series for their walkthrough for their write ups of the labs but I want to cram this all into one video so look they get into the good stuff we're setting up a seam and you could also toggle on rules to alert us when Defenders are attacking our organization what tradecraft what ttps from the miter attack framework and all are they all up to but this is awesome you can get started with elk using the elastic Cloud just 14-day trial doesn't require a credit card you just need an email and a password and all we do is just set up a free account so I'm going to do it jumping over to this URL this is all it takes just start your free IC Cloud trial let me fill out my email address choose a password and then sign up with email nice and easy now we can just cruise through a super simple form hey I'll just put my name company is self uh I am new to elastic and I'm more interested in security I'd like to just learn more about elastic let's do it all right now we need to create a new deployment I can just call mine I don't know security deployment how about that uh we could change some of the settings but I think I'm just fine with the defaults let's go and create our deployment and cool oh w we have 150 days left of our trial goodness it's more than 14 okay now it's doing its thing it is creating our deployment doing whatever configuration things that it needs we could cruise through with the tour um but I don't really need to do that I just kind of want to go back to my deployment um oh shoot and it showed me credentials can I get back to that these root credentials are shown only once oh goodness okay uh I guess I'll just check the frame of the video maybe and it is still creating the deployment the video is cruising through but I have now seen after a little bit of time the Cabana menu open up in the navigation so kind of taking a look at what the lab suggests we should be able to go ahead and open up Cabana and once this thing finishes up we can go ahead and move on with the lab here okay now this has popped up looks like I have my cabana instance up and running um I can edit the configuration I can play with monitoring the health here copy endpoint can I just open this oh okay cool yeah now we're going somewhere new all right now we've loaded up Cabana seemingly or we're still in elastic but let me go ahead and manage deployment and I could move down to okay Security Management o fleet Fleet is what I'm looking for that is what I suggested next in the lab and we want to be able to add an agent here so I'm going to go ahead and click on this add agent button and then adding elastic agents to your hosts allows it to collect data and send it to the elastic stack okay what type of host are you adding they're controlled by an agent policy creating new policy to get started um I realize my face is in the way uh the Advanced options no I think that's all just fine I'm going to assume again totally defaults are good I'll H create policy and then we'll be able to allow the other options to enroll in Fleet and install the elastic agent will all be done for me cool yep okay seemingly good we will enroll in Fleet install the elastic agent on your host oh okay we will toggle this to Windows and that should be all good for me I'll just want to copy this syntax and then the lab suggests hey we just save this we just take note of it so we know how we can go ahead and install this when the time comes but then we'll move into part two of this little lab walkthr and that way we'll be able to actually install and configure the elastic agent so let me just open up notepad I suppose that's fine and I'll paste this in so it looks like this syntax like the Powershell code that they give here is just everything that you need to actually download the elastic agent expand the archive like decompress the zip file and then install the elastic agent uh I think we could basically skip over what would be lab number two here on installing the whole agents so let me go ahead and copy the syntax and I'll open up a Windows terminal I'll h control shift enter on my keyboard so that I can open this up in the admin mode I'm going to go and full screen this and I suppose I will make a directory for like elastic so at least this is kind of clean and not just randomly in my user profile now I'll go ahead and paste all this in because there's currently nothing in the path here and I'll let it download the elastic agent for me now that that's done it's going to try and decompress the zip archive expand archive and Powershell okay and now it's going to go ahead and install the agent it says the elastic agent will be installed in C program files elastic agent and will run as a service do you want to continue let's hit y for yes enter that and let it do its thing okay it took a little bit but uh looks like it says successfully triggered restart on running elastic agents successfully enrolled the elastic agent the elastic agent has been successfully installed awesome let me clear the screen here toggling back over to elastic over in the web browser you can see hey One agent has been enrolled incoming data is confirmed and we are ingesting everything that we need we can click on that view enrolled agent and here it is there's my desktop host name now I can click on this and go take a look at what is all coming from this here's the last activity last check-in message agent policy that we Define the agent version platform okay so now in the intro laabs walkthrough we basically just jumped over what would be part two and now we can move on to part three where we're chatting about what data we might ingest into elastic and they say look by default Windows logs are not ideal because it's just kind of aorus Borg of whatever actually comes through for it and some things might not actually be audited by default so to get logs that are more readable and useful we can use and we should be using cismon by the way you'll practically like never ever find a client organization and environment that is actually using in as deployed cismon but when you do if you do it's awesome we can follow this link to download sysmon it is part of the tool sets that are created by Mark rosovich let me open this up in a new tab here I can scroll down and click the download cismon and now I do have that zip archive once more let's move back to our uh administrative Powershell window and move into the downloads directory oh forgive me that should be downloads and I know look yeah I could probably do this all in one command but I just like typing CD over and over again uh so let's get our cismon doz file that I see there let's go ahead and expand archive just as we saw in the elastic agent syntax to go ahead and extract this ZIP archive and now we should have a sysmon directory as we do so let's move in into that directory and I have the cismon 64 that we probably want to run on our 64-bit architecture we can go ahead and run our cismon 64.exe failed to start the service the operation completed successfully what does that mean uh what does the lab suggest okay they uh end up using cismon on its own Tac I Tac n and accept Ula is Tac I to install is there like a tack H for help yeah okay cool okay the usage we can install with cismon Tac I what is n was that even a thing uh it doesn't seem to be anymore anyway so let me use that cismon 64 Tac I cismon is already registered uninstall cismon before reinstalling okay so we're good like it's just doing its thing right now can I get service oh yeah yeah yeah okay there is 64bit uh cismon running as a service so I'm assuming all is good and now that cismon is running on our system we need to configure our elastic agent to configure and gather these logs sign into your account navigate back to Cabana move into Fleet and then check out the Integrations as to what agents might be pulling stuff in then we can add the integration for Windows and then toggle on the button for sysmon uh let's go try it out so back in Cabana as part of our elk stack we'll move over to Fleet and I don't see any Integrations oh oh oh oh if we go into agent policies you can click in on the policy that you've defined and now the Integrations is there let me see if I can add integration and I'm going to assume I would be able to browse for Windows there's a whole lot of entries here uh let me just go and search for it let me search for Windows here we go click on Windows I just want to scroll down into this overview does it actually give me a little bit more like sysmon specifically I don't know let's try it let me just add Windows there we go and uh integration name is Windows one forwarded Powershell Powershell operational oh syst one operational okay perfect I think all of this looks good we can add it to existing hosts with the agent policy one and let me click the bottom right button that my face is in the way save and continue save and deploy changes I'm good with that okay Windows one integration added now our agent policy one has system integration and windows perfect uh let me go take a look back at our Fleet let's check our agents and we should see that it is working with the windows integration and can pull from uh sysmon just as well now it says Hey play around on the computer that has the elastic agent installed move files around create file Start program make a few Google searches this will generate some LS to ensure we have syst on logs reaching our Cloud after you've created some log activities you can navigate to Cabana discover well okay uh let me get back to I suppose our little command line here let's just fire up the calculator of course that normal operations can I run like who am I I don't know if that'll do anything um I don't know should I just open up word pad how about that is that going to run is it in the path how do you access word pad Powershell probably just didn't know where the heck it was whatever uh so so hopefully we have some Sison log events now I think uh Sison process start is just one when you've created a process uh the event ID for cismon is one so if we navigate back to Cabana move into the Discover dashboard set the source to logs then we can look at the time constraint for today uh let me go back to the little hamburger menu and let's go to discover let's set our uh data view source to logs we'll set this to today as it is is good and now I need to go figure out and find what Fields would be worthwhile to search for uh our agent name is probably worth while because I want to get the things from our desktop good and if I put this in the documents view then it'll actually show it with the timestamp uh can I get any specific like process names that are started we have a vent action that might be worth adding okay not a whole lot of entries there DNS queries interesting O process create process create that is good that's got to be an event ID that comes with that right okay event ID let me add this a lot of those are empty even on process create so that's dumb are there any processes that we can run oh even Powershell stuff though that could be worthwhile process O Okay process command line let me add this okay now can I see us trying to run oh yeah I can here's my word pad excellent here's who am I as I just type those in the command line and Cal check it out here's us trying to run cismon oh the lab actually says you can set a filter on your data to limit the results just to Sison data that can be done by setting the data stream. datet field for windows. cismon operational uh okay we can try that okay so add filter um we wanted data stream. dat set is and then windows. syston operational right let's add filter Okay cool so it was looking at the same sort of stuff we were looking at just a moment ago and check it out there is our process create word pad who am I in Cal nice so if we wanted to filter that even more I think we could do like uh what is it it's win log event ID can be uh colon one right so it's setting to a value of one and that should be the I don't I don't want an and I just want that please can I do that go filter yeah okay so now we're only getting the process create and you can see cismon you can see uh elastic stack and the agent coming together that is super duper cool and that can help us do some further analysis with an elk and that is that that is three of the kind of written GitHub free Labs part of the introductory courses of John strand anti- siphon training Black Hills information security all of their pay what you can courses and really really cool that we finally just got an opportunity to spin up elk because now we can do a little bit more of that you know sweet stuff detection engineering I don't know tracking around in an EDR and a seam to see what logs are happened where when and how all the stuff that can help you for your job and like the real world in the industry I hope that's pretty cool I hope that is actually tactical uh information security education so hey check out Black Hill information security antiphon training pay what you can courses all the incredible stuff that John Str is up to and thank you so much for watching this video hope it was fun hope you learned something new hope we had a great time together and I'll see you in the next video like comment subscribe become a member become a member of the channel that really really helps support all the stuff that we're doing here thanks again