Tech note: The right way to handle false positives

June 27, 2016

False positives are a fact of life when dealing with even the best Cyber Threat Intelligence (CTI) sources. This Tech Note will show you one way to manage these so that you can resolve them locally and communicate to the provider of the CTI suggested improvements to fix the issue for all of the recipients.

Sometimes you get CTI data that has one or more observables that don’t map to threat activity and generate a lot of noise in detection tools for no seemingly good reason. The first thing you need to do is confirm this is a false positive. Please see the blog post the “truth about false positives” for some ways to dig into CTI to asses on what is truly a false positive vs. lack of sufficient context.

Let us walk through an example from Soltra’s own systems where we had a match on our end point security tool Confer with CTI data from both the FS-ISAC and HailAtaxii on the same Observable.

Just a quick aside, the way Confer’s STIX integration works is that we build a feed from our Soltra Edge instance to an applet (built by Confer) that we are running on a server in Soltra’s infrastructure. The applet also connects with Confer’s web app which pulls the end point activity of our workstations. The CTI data from our Soltra Edge instance is compared against the activity seen on the workstations. When there are matches these are pushed back from the applet to the Confer web dash board where we are alerted of suspicious activity.

So in our example we had many hits to the IP address 172.255.83.2, which when researched was found to be the server for a Certificate Revocation List (CRL) for several certificate authorities. The logs in the Confer end point monitoring software showed that a number of workstations were accessing the destination from the CTI reports.

In this case we know that the “oscpd” executable on MacOS is what checks certificate validation by connecting to the certificate revocation list for the certificate in question. The domain name crt.comodoca.com also aligns. We then looked at some enrichment sources for domain history (we used passivetotal.com) and confirmed that this longstanding domain/IP maps to several CRLs. We also know that the TTPs related to each of the indicators were run-of-the-mill criminal malware and not a highly advanced threat actor that might be using the CRL as an On/Off switch to control activity of a compromised host. (Such as check Cert if valid do X if revoked do Y). With all of this research and analysis in hand we have reasonable confidence that these appearances of the Observables are false positives.

How to “remove” false positive Observables

If you are sure an observable (or any STIX object for that matter) is a false positive, here is the best way to resolve it in Soltra Edge 2.8.x

  1. Go into the Admin screen and create a “Stix Activity” in https://your-edge-server/activity/

  2. For this example we created two “STIX Activity Actions” for different reasons that a false positive crept into the CTI data. STIX Activity actions are local to a specific Edge instance and are not shared externally. These definitions are setup by the administrator, but can be assigned by any user to specific STIX objects in the user interface.

  3. Use the search function to find the Observables of concern

  4. Assign the STIX activity action to this observable (or others with the same IP). This way, if the Observables reappear in our system later as the upstream producer edits them or resends them, we already know that we have researched and categorized it. (In this example we had manually downloaded them while building this example and then re-imported them which is why you see a more recent date. When they were re-imported the STIX Activity Actions were relinked so we already knew what we were dealing with and didn’t have to reinvestigate)

  5. The easiest way to do this is to open each observable in a new browser tab and leave a tab open to the search screen. Scroll down to the bottom of the Observable view screen and assign the STIX Activity action to the Observable. (you will need to do this to for each item)

  6. Add some descriptive text on the comments so that you can record the history of why you concluded this is a false positive.

  7. At this point review the observables and make sure the STIX Activity Action is shown.

    • We then cut & paste that text and emailed it to the upstream producer of the CTI data that included this observable, so that they can edit the content, and make a new version of their STIX objects to be shared out with everyone who received them.

    • We include the Indicator ID (fsisac:observable-66a9591a-e740-48e8-b7e3-bb8c42dff0be) for this example,

    • the data value in question (178.255.83.2), and:

    • our rationale of why it should be edited/removed (the text from the comment)

    • At present This is the best method to get this fixed. (Yes, we see this as a Soltra Edge product feature enhancement opportunity).

  8. We then select the observables and move them to the Clipboard. You may also want to download them to your local workstation just in case.

  9. We now see in our example 3 objects in the clipboard (see the upper right corner of the screen)

  10. Click on the Clipboard icon to go to the batch update screen.

  11. The click on the “Delete” tab.

  12. Click on the Delete button on the lower right hand side of the screen. Confirm OK and now these Observables are removed from your system’s active data sets. The deleted Observables will not show up in search results for Feeds anymore.

Important Note: if the upstream provider updates the Indicator to which these are correlated or there is re-sync with that data source these Observable items may return. That is why it is important to both use the STIX Activity Action to mark them with your research results and to communicate with the producer to make the fix.