User guide of MISP Malware Information Sharing

User guide of MISP Malware Information Sharing
Table of Contents
Introduction
1.1
Book Convention
1.2
Quick Start
1.3
Get Your Instance
1.4
General Layout
1.5
General Concepts
1.6
User Management and Global Actions
1.7
Using the System
1.8
Delegation of Event
1.9
Administration
1.10
Managing Feeds
1.11
Automation and MISP API
1.12
PyMISP - Python Library to Access MISP
1.13
Create an Event Based on a Report
1.14
Managing Feeds
1.15
Taxonomies
1.16
Galaxies
1.17
Sightings
1.18
Categories and Types
1.19
Synchronisation/Sharing
1.20
ZeroMQ - MISP publish-subscribe
1.21
Appendices
1.22
Introduction
User guide for Malware Information Sharing Platform (MISP) - A Threat Sharing Platform. This user
guide is intended for ICT professionals like security analysts, security incident handler, or malware
reverse engineers who share threat indicators using MISP or integrate MISP into other security
monitoring tools. The user guide includes day-to-day usage of the MISP's graphical user interface
along with its automated interfaces (API), in order to integrate MISP within a security environment.
Acknowledgement
The MISP user guide is a collaborative effort between all the contributors to MISP including:
Belgian Ministry of Defence (CERT)
CIRCL Computer Incident Response Center Luxembourg
Iklody IT Solutions
NATO NCIRC
Cthulhu Solutions
CERT-EU
and many other contributors especially the ones during the MISP hackathons.
Contributing
We welcome contributions to the MISP book. If you want to contribute, clone the misp-book repository
and pull a request with your changes. You can also open issues if you find any errors or propose
changes.
License
The MISP user guide is dual-licensed under GNU Affero General Public License version 3 and CCBY-SA 4.0 international.
Copyright (C) 2012 Christophe Vandeplas
Copyright (C) 2012 Belgian Defence
Copyright (C) 2012 NATO / NCIRC
Copyright (C) 2013-2017 Andras Iklody
Copyright (C) 2015-2017 Alexandre Dulaunoy
Copyright (C) 2014-2017 CIRCL - Computer Incident Response Center Luxembourg
Convention Used in This Book
code block or value
Used for variable, function or menu names in MISP.
Example Organisations
As MISP is a platform to support information sharing, example organisations are often used within this
book.
A set of users and organisations are used in the different examples.
The following two organisations are regularly used as example:
Setec Astrononomy with UUID
Acme Finance with UUID
58d38339-7b24-4386-b4b4-4c0f950d210f
58d38326-eda8-443a-9fa8-4e12950d210f
Starting from MISP 2.4.71, the example organisations with the above mentioned UUID are black-listed
to avoid large distribution of sample events while testing a MISP instance. If you want to test your
distribution, the sample organisation black-listing can be removed in
Blacklists
.
Administration
under
Manage Org
Quick Start
The Malware Information Sharing Platform (MISP) tool facilitates the exchange of Indicators of
Compromise (IOCs) about targeted malware and attacks, within your community of trusted members.
MISP is a distributed IOC database containing technical and non-technical information. Exchanging
such information should result in faster detection of targeted attacks and improve the detection ratio,
whilst also reducing the number of false positives.
Create an Event
You only have to add a few pieces of information to register your Event. Further details will be
specified after the Event has been added.
Describe Event
Red is totally normal. No worries.
Now you can specify the information for your Event (you will need to scroll the window).
Free-Text Import Tool
If you have a list of indicators from which you would like to quickly generate attributes then the Freetext import tool is just what you need. Simply paste your list of indicators (separated by line-breaks)
into this tool.
The tool will help you to find similarities between your import and other issues already registered in
MISP.
For example, you can see the ID of all related Events and view their information.
Tags and Taglist
Using existing Data
Another easy way to add information is to use Tags. You can see the result of adding existing Tags
(circl:incident-classification=XSS ans circl:incident-classification="information-leak).
By clicking the button, you can add more tags from an existing Taglist.
In particular the "Taxonomy Library: circl" Taglist is very complete, as you can see:
Make your own Taglist
If you want make your own Taglist, select Add Tag.
You will see the following window:
Then, when you add the new tag it will appear in the Custom Taglist.
Suggestions
The following attribute types should be added for each Event:
ip-src: source IP of attacker
email-src: email used to send malware
md5/sha1/sha256: checksum
Hostname: full host/dnsname of attacker
Domain: domain name used in malware
Browsing Events
To see your Event, select List Events from the menu Events Action. You can click any row and select a
filter.
If you click on your Event's number, you can see all the information related to your Event.
Export Events for Log Search
Export functionality is designed to automatically generate signatures for intrusion detection systems.
To enable signature generation for a given attribute, the Signature field of this attribute must be set to
Yes. Note that not all attribute types are applicable for signature generation, currently we only support
NIDS signature generation for IP, domains, host names, user agents etc., and hash list generation for
MD5/SHA1 values of file artifacts. Support for more attribute types is planned.
Simply click on any of the following buttons to download the appropriate data for log correlation.
Get your own MISP instance
The intention of this chapter is to support you in getting your own MISP instance up and running.
MISP Virtual Machine
CIRCL maintains the image of a recent MISP virtual machine online.
This is a very easy out of the box solution, optimized for product evaluation and to support trainings
hold by CIRCL staff.
The images is updated on a regular base. You should frequently re-visit the online resources to get
the latest versions including bug fixes and new features.
MISP VM Download
The best place to get the latest version of the MISP virtual machine, as well as all the available
training materials is the MISP training materials page on the CIRCL website.
If you do not remember the direct link to the MISP training materials here are the very easy to
remember step you have to follow to reach the right place:
1. Access the CIRCL homepage
2. Navigate to the Training area
3. Click MISP Malware Information Sharing Platform - Threat Sharing
4. Follow the link to the Training materials freely available
Download the image of the virtual machine and validate the SHA512 fingerprint.
Import Appliance
In VirtualBox use the "Import Appliance..." functionality to import the virtual machine.
The instructions in this manual covers VirtualBox only. If you prefer another virtualization solution like
VMWare you can find some quick instruction on the MISP training materials page.
MISP VM Credentials
The MISP image is pre-configured to be reachable on the private IP address 192.168.56.50 by SSH.
The GUI is reachable by http://192.168.56.50/.
MISP credentials:
GUI User: [email protected] : Password1234
GUI Admin: [email protected] : Password1234
Shell/SSH: misp : Password1234
MySQL User: misp : Password1234
MySQL Root: root : Password1234
Potential issues
During life trainings we see in rare cases that some users could not reach the virtual machine over the
virtual network.
Some investigations discover that this always happens with user whom already had VirtualBox in use
before and had already one or more Host-only Adapter configured in advance.
The MISP image is pre-configured to use Host-only Adapter with the Name vboxnet0.
If this is already occupied by previous VirtualBox projects, try to attach the network adapter to the next
available Host-only network.
General Layout
The top bar
Simple User
This menu contains all of the main functions of the site as a series of dropdown menus. These
contains all (from the current simple user's perspective) accessible functions sorted into several
groups.
Home button: This button will return you to the start screen of the application, which is the event
index page (more about this later).
Event Actions: All the malware data entered into MISP is made up of an event object that is
described by its connected attributes. The Event actions menu gives access to all the functionality
that has to do with the creation, modification, deletion, publishing, searching and listing of events
and attributes.
Input Filters: Input filters alter what and how data can be entered into this instance. Apart from
the basic validation of attribute entry by type, it is possible for the site administrators to define
regular expression replacements and blacklists for certain values in addition to blocking certain
values from being exportable. Users can view these replacement and blacklist rules here whilst
administrator can alter them.
Global Actions: This menu gives you access to information about MISP and this instance. You
can view and edit your own profile, view the manual, read the news or the terms of use again, see
a list of the active organizations on this instance and a histogram of their contributions by attribute
type.
Discussions: Link to the discussion threads.
Admin Menu Bar
Home button: idem as user.
Event Actions: ibidem
Input Filters: Ibidem
Global Actions: Ibidem
Sync Actions: With administrator access rights, shows a list of the connected instances and
allows the initiation of a push and a pull (more about the synchronization mechanisms later).
Administration: Administrators can add, edit or remove user accounts and user roles. Roles
define the access rights to certain features such as publishing of events, usage of the REST
interface or synchronization of any user belonging to the given role. Site administrators can also
access a contact form, through which it is possible to reset the passwords of users, or to just get in
touch with them via encrypted e-mails.
Audit: If you have audit permissions, you can view the logs for your organization (or for site
admins for the entire system) here or even search the logs if you are interested in something
specific.
Proposal Notifications: This shows how many proposals your organization has received and
across how many events they are spread out. Clicking this will take you to the list of proposals.
Log out: Logs you out of the system.
A list of the contents of each of the above drop-down menus
Event actions
List Events: Lists all the events in the system that are not private or belong to your organisation.
You can add, modify, delete, publish or view individual events from this view.
Add Event: Allows you to fill out an event creation form and create the event object, which you
can start adding attributes.
List Attributes: Lists all the attributes in the system that are not private or belong to your
organisation. You can modify, delete or view each individual attribute from this view.
Search Attributes: You can set search terms for a filtered attribute index view here.
View Proposals: Shows a list of all proposals that you are eligible to see.
Events with proposals: Shows all of the events created by your organisation that has pending
proposals.
List Tags: List all the tags that have been created by users with tag creation rights on this
instance.
Add Tag: Create a new tag.
List Templates: List all of the templates created by users with template creation rights on this
instance.
Add Template: Create a new template.
Export: Export the data accessible to you in various formats.
Automation: If you have authentication key access, you can view how to use your key to use the
REST interface for automation here.
Input filters
Import Regexp: You can view the Regular Expression rules, which modify the data that can be
entered into the system. This can and should be used to help filter out personal information from
automatic imports (such as removing the username from windows file paths), having unified
representation for certain common values for easier correlation or simply standardizing certain
input. It is also possible to block certain values from being inserted. As a site administrator or a
user with regex permission, you can also edit these rules.
Signature Whitelist: You can view the whitelist rules, which contains the values that are blocked
from being used for exports and automation on this instance. Site administrators have access to
editing this list.
List Warninglists: MISP warninglists are lists of well-known indicators that can be associated to
potential false positives, errors or mistakes. The warning lists are integrated in MISP to display an
info/warning box at the event and attribute level.
Global Actions
News: Read about the latest news regarding the MISP system
My Profile: Manage your user account.
Dashboard: allow you to see your Notifications of Proposals, Events with proposals and
Delegation request. Your can see the last changes since your last visit, as Events updates and
Events publications.
Members List: View the number of users per organization and get some statistics about the
currently stored attributes.
Organizations: View the organizations having a presence on this instance, with some useful
informations as contact's name.
Role Permissions: You can view the role permissions here.
List Sharing Groups: You can view the list of existing Sharing Groups who you or your
organization have access.
Add Sharing Group: You can create a sharing group.
User Guide: A link to this user guide.
Statistics: View a series of statistics about the users and the data on this instance.
Sync Actions
List Servers: Connect your MISP instance to other instances, or view and modify the currently
established connections. It may be that you have an Error Message in the page (if you enabled
debug or site_admin_debug settings). An example of error message:
An easy first them to make most of them go away is to use the clean cache feature on the server
settings menu, diagnostics tab.
You must then scroll down the page.
List Feeds: Follow the RSS feeds of other organization or CERTs worldwide.
Administration
List Users: View, modify or delete the currently registered users.
New User: Create an account for a new user for your organisation. Site administrators can create
users for any organisation.
Contact Users: You can use this view to send messages to your current or future users or send
them a temporary password.
When adding a new user to the system, or when you want to manually reset the password for a user,
just use the "Send temporary password" setting.
After selecting the action, choose who the target of the e-mails should be (all users, a single user or a
user not yet in the system).
You can then specify (if eligible) what the e-mail address of the target is (for existing users you can
choose from a dropdown menu).
In the case of a new user, you can specify the future user's GPG key, to send his/her new key in an
encrypted e-mail.
The system will automatically generate a message for you, but it is also possible to write a custom
message if you tick the check-box, but don't worry about assigning a temporary password manually,
the system will do that for you, right after your custom message.
List Organizations: View the organizations having a presence on this instance, with some useful
informations.
Add Organization:
List Roles: List, modify or delete currently existing roles.
Add Role: Create a new role group for the users of this instance, controlling their privileges to
create, modify, delete and to publish events and to access certain features such as the logs or
automation.
Administrative Tools: Various tools, upgrade scripts that can help a site-admin run the instance.
Server Settings: Set up and diagnose your MISP installation.
Jobs: View the background jobs and their progress
Scheduled Tasks: Schedule the pre-defined tasks for your instance (this currently includes
export caching, server pull and server push).
Audit
List Logs: View the logs of the instance.
Search Logs: Search the logs by various attributes.
Discussions
List Discussions: List all of the discussion threads.
Start Discussion: Create a new discussion thread.
The left bar
This bar changes based on each page-group. The blue selection shows you what page you are on.
General Concepts
Admins and Site Admins
Background Jobs
MISP Instance
Organisation administrators and Site administrators
Pivot path
Pivoting
Proposals
Publishing
Pull
Push
Scheduled Tasks
Sync User
Synchronisation
Tagging
Templating
General Concepts
Admins and Site Admins
There are two types of admins in MISP: Admins (also refered to as org admins) and Site Admins.
Whilst the former can only do some limited administration of users of his/her own organisation, site
admins have access to all of the features and data of the system. They are in charge of making sure
that the system runs correctly and the maintenance of MISP.
Background Jobs
A lot of the heavier tasks are a burden to users, in that their actions can cause long delays (and in
some cases timeouts) while the application logic is executing. To alleviate this, long processes have
been (if enabled) moved to background jobs, meaning that their execution happens asynchronously
in the background, allowing the user to freely interact with the platform whilst the request is being
processed.
MISP Instance
A MISP instance is an installation of the MISP software and the connected database. All the data
visible to the users is stored locally in the database and data that is shareable (based on the
distribution settings) can be synchronised with other instances via the Sync actions. The instance that
you are reading this manual on will be refered to as "this instance" or "your instance". The instances
that your instance synchronises with will be refered to as "remote instances".
Organisation administrators and Site administrators
We have two types of administrators, site and organisation admins. The former has access to every
administrator feature for all the data located on the system including global features such as the
creation and modification of user roles and instance links, whilst organisation admins can administer
users, events and logs of their own respective organisations.
Pivot path
The (branching) path taken by a user from event to event while following correlation links. This is
represented by the branching graph in the event view.
Pivoting
The act of navigating from event to event through correlation links.
Proposals
Each event can only be directly edited by users of the original creator organisation (and site admins).
However, if another organisation would like to amend an event with extra information on an event, or
if they'd like to correct a mistake in an attribute, they can create a Proposal. These proposals could
then be accepted by the original creator organisation. These proposals can be pulled to another
server, allowing users on connected instances to propose changes which then could be accepted by
the original creators on another instance (and subsequently pushed back).
Publishing
When an event is first created by a user, it is visible to everyone on the instance based on the access
rights ("Your organisation only" events will not be visible to users of other organisations), but they will
not be synchronised and they won't be exportable. For this, a user with publishing permission of the
organisation that created the event has to publish the event. The system will then inform all the users
of the instance that are subscribing to e-mail notifications and who have access to view the published
event via an e-mail.
Pull
Pulling is the process of using the configured sync user on a remote instance to REST GET all of the
accessible data (based on the distribution rights) to your instance and store it.
Push
Pushing is the process of using a configured instance link to send an event or all accessible events
(limited by the distribution rights) through the REST interface to a remote instance.
Scheduled Tasks
Certain common tasks can be scheduled for a later execution or for regular recurring executions.
These tasks currently include caching all of the export formats, pulling from all eligible instances and
pushing to all eligible instances.
Sync User
A user of a role that grants sync permissions, these users (and their authentication keys) are used to
serve as the points of connection between instances. Events pushed to an instance are pushed to a
sync user, who then creates the events on the remote instance. Events pulled are added by the sync
user that is used to connect the remote instance to your instance. As an administrator, keep in mind
that a sync user needs auth key and publish permissions, has to have undergone the mandatory
password change and has to have accepted the Terms of Use in order for the sync to work. Please
make sure that all of these steps are taken before attempting to push or pull.
Synchronisation
What we call synchronisation is an exchange of data between two (or more) MISP instances through
our pull and push mechanisms.
Tagging
Users with tagging rights can assigned various dynamically created tags to events, allowing an
arbitrary link between events to be created. It is possible to filter events based on these tags and they
can also be used to filter events for the automation.
Templating
Users with templating rights can create easy to fill forms that help with the event creation process.
User Management and Global Actions
First run of the system
User Management and Global Actions
First run of the system
When first logging into MISP with the username and password provided by your administrator, there
are a number of things that need to be done, before you can start using the system.
Acceping the Terms of use: The terms of use are shown immediately after logging in for the first
time, make sure to read through this page before clicking "Accept Terms" at the bottom of the
page.
Changing the password: After accepting the ToU, you'll be prompted to change your password,
but keep in mind that it has to be at least 6 characters long, it has to include at least one uppercase and one lower-case character in addition to a digit or a special character. Enter the same
password into the confirm password field, before clicking submit to finalise the change.
Setting up the GPG Key: In order for the system to be able to encrypt the messages that you
send through it, it needs to know your GPG key. Navigate to the Edit profile view (My Profile on
the left -> Edit profile in the top right corner). Paste the key into the Gpgkey field and click submit.
Subscribing to Auto-alerts: Turning auto-alerts on will allow the system to send you e-mail
notifications about any new public events entered into the system by other users and private
events added by members of your organisation. To turn this on, navigate to the Edit profile view
(My profile on the left navigation menu -> Edit profile in the top right corner). Tick the auto-alert
checkbox and click submit to enable this feature.
Subscribing to e-mails sent via the "Contact Reporter" functionality: This feature is turned on
right below the autoalerts and will allow you to receive e-mails addressed to your organisation
whenever a user tries to ask about an event that was posted by a user of your organisation. Keep
in mind that you can still be addressed by such a request even when this setting is turned off, if
someone tries to contact you as the event creator directly or your organisation for an event that
you personally have created then you will be notified.
Reviewing the Terms & Conditions: To review the Terms & Conditions or to read the User
Guide, use the appropriate button on the left navigation menu.
Making sure that compatibility mode is turned off (IE9&IE10): Compatibility mode can cause
some elements to appear differently than intended or not appear at all. Make sure you have this
option turned off.
Using the system
Creating an event:
Add attributes to the event:
Add Attribute
Create and manage Sharing Groups
Populate from Template
Freetext Import Tool
Attribute Replace Tool
Add attachments to the event:
Propose a change to an event that belongs to another organisation
Populate from OpenIOC
Populate from ThreatConnect
Adding IOCs from a PDF report
Publish an event:
Browsing past events:
To list all events:
Filters
Event view
General Event Information
Event History:
Listing all attributes:
Searching for attributes:
Updating and modifying events and attributes:
Tagging:
Templating:
Contacting the reporter:
Automation:
Exporting data:
Export page with background jobs disabled
Export page with background jobs enabled
Exporting search results and individual events
Connecting to other instances:
Setting up a connection to another server:
Browsing the currently set up server connections and interacting with them:
Rest API:
Requests
Example - Get single Event
Example - Add new Event
Using the system
Creating an event:
The process of entering an event can be split into 3 phases, the creation of the event itself, populating
it with attributes and attachments and finally publishing it.
During this first step, you will be create a basic event without any actual attributes, but storing general
information such as a description, time and risk level of the incident. To start creating the event, click
on the New Event button on the left and fill out the form you are presented with. The following fields
need to be filled out:
Date: The date when the incident has happened. Just click this field and a date-picker will pop up
where you can select the desired date.
Distribution: This setting controls, who will be able to see this event once it becomes published
and eventually when it becomes pulled. Apart from being able to set which users on this server
are allowed to see the event, this also controls whether the event will be synchronised to other
servers or not. The distribution is inherited by attributes: the most restrictive setting wins. The
following options are available:
Your organization only: This setting will only allow members of your organisation to see this.
It can be pulled to another instance by one of your organisation members where only your
organisation will be able to see it. Events with this setting will not be synchronised. Upon
push: do not push. Upon pull : pull.
This Community-only: Users that are part of your MISP community will be able to see the
event. This includes your own organisation, organisations on this MISP server and
organisations running MISP servers that synchronise with this server. Any other
organisations connected to such linked servers will be restricted from seeing the event. Upon
push: do not push. Upon pull: pull and downgrade to Your organization only.
Connected communities: Users that are part of your MISP community will be able to see the
event. This includes all organisations on this MISP server, all organisations on MISP servers
synchronising with this server and the hosting organisations of servers that connect to those
afore mentioned servers (so basically any server that is 2 hops away from this one). Any
other organisations connected to linked servers that are 2 hops away from this own will be
restricted from seeing the event. For more information on community-related distribution
levels, click here. Upon push: downgrade to This Community only and push. Upon pull: pull
and downgrade to This Community only.
All communities: This will share the event with all MISP communities, allowing the event to
be freely propagated from one server to the next. Upon push: push. Upon pull: pull.
Sharing group: This will share the event to the defined sharing group. This includes only the
organisations defined in the sharing group. The distribution can be local and cross-instance
depending of the sharing group definition. For more information on sharing groups, refer to
the sharing group section.
Threat Level: This field indicates the risk level of the event. Incidents can be categorised into
three different threat categories (low, medium, high). This field can alternatively be left as
undefined. The 3 options are:
Low: General mass malware.
Medium: Advanced Persistent Threats (APT)
High: Sophisticated APTs and 0day attacks.
Analysis: Indicates the current stage of the analysis for the event, with the following possible
options:
Initial: The analysis is just beginning
Ongoing: The analysis is in progress
Completed: The analysis is complete
Event Description: The info field, where the malware/incident can get a brief description starting
with the internal reference. This field should be as brief and concise as possible, the more
detailed description happens through attributes in the next stage of the event's creation. Keep in
mind that the system will automatically replace detected text strings that match a regular
expression entry set up by your server's administrator(s).
GFI Sandbox: It is possible to upload the exported .zip file from GFI sandbox with the help of this
tool. These will be dissected by the MISP and a list of attributes and attachments will
automatically be generated from the .zip file. Whilst this does most of the work needed to be done
in the second step of the event's creation, it is important to manually look over all the data that is
being entered.
Add attributes to the event:
The second step of creating an event is to populate it with attributes and attachments. This can be
done by adding them manually or importing the attributes from an external format (OpenIOC,
ThreatConnect). To import from an external format or to upload an attachment use the options in the
menu on the left.
Using the above shown buttons, you can populate an event using various tools that will be explained
in the following section. Let's start with the Add Attribute button.
Add Attribute
Keep in mind that the system searches for regular expressions in the value field of all attributes when
entered, replacing detected strings within it as set up by the server's administrator (for example to
enforce standardised capitalisation in paths for event correlation or to bring exact paths to a
standardised format). The following fields need to be filled out:
Category: This drop-down menu explains the category of the attribute, meaning what aspect of
the malware this attribute is describing. This could mean the persistence mechanisms of the
malware or network activity, etc. For a list of valid categories, click here
Type: Whilst categories determine what aspect of an event they are describing, the Type explains
by what means that aspect is being described. As an example, the source IP address of an attack,
a source e-mail address or a file sent through an attachment can all describe the payload
delivery of a malware. These would be the types of attributes with the category of payload deliver.
For an explanation of what each of the types looks like together with the valid combinations of
categories and types, click here
Distribution: This drop-down list allows you to control who will be able to see this attribute. The
distribution is inherited by attributes: the most restrictive setting wins. For more info, read the
distribution information in the creating an event section - click here
Contextual Comment: Add a comment to the attribute. This will not be used for correlation.
Value: The actual value of the attribute, enter data about the value based on what is valid for the
chosen attribute type. For example, for an attribute of type ip-src (source IP address), 11.11.11.11
would be a valid value. For more information on types and values, click here
Contextual Comment: You can add some comments to the attribute that will not be used for
correlation but instead serves as purely an informational field.
For Intrusion Detection System: This option allows the attribute to be used as an IDS signature
when exporting the NIDS data, unless it is being overruled by the white-list. For more information
about the whitelist, head over to the administration section.
Batch import: If there are several attributes of the same type to enter (such as a list of IP
addresses, it is possible to enter them all into the same value-field, separated by a line break
between each line. This will allow the system to create separate lines for the each attribute.
Create and manage Sharing Groups
Sharing groups in MISP are a more granular way to create re-usable distribution lists for
events/attributes that allow users to include organisations from their own instance (local
organisations) as well as organisations from directly, or indirectly connected instances (external
organisations). Sharing groups can be created by any user that has the sharing group editor
permission. Additionally, sharing groups can be edited by any user that has the aforementioned
permission in addition to being a member of the sharing group's creating organisation, or any
organisation that is marked as an "extender" of the sharing group. The main use for the extend feature
is delegating the rights to add users to trusted partners. For example, when sharing with a different
industry sector, knowing all actors that should receive the information is often not possible, so
delegating the rights to extend the event to a trusted representative of said sector would allow for
someone with more insight to find and add the proper list of partners for the sharing group.
The most general use-cases for sharing groups are creating re-usable topical subgroups in MISP that
share events or for ad-hoc sharing scenarios (such as several organisations involved in a specific
incident wanting to work together). Generally sharing groups add a level of complexity for the users
involved as well as a performance overhead on the data marked with it.
As a best-practice recommendation, using traditional distribution methods is prefered unless they
cannot cover the given use-case. Also, whilst sharing groups can be assigned to both events and
attributes, it is highly recommended to use the special "inherit" distribution setting on attributes
whenever the attribute's sharing group would match the event's.
Sharing groups consist of the following elements, each of which has its own page in the sharing
group creator/editor tool (accessed via the Global actions -> List Sharing Groups and Add Sharing
Group functionalities):
General: Metadata describing the intent of the sharing group
Name: The unique name of the sharing group.
Releasable to: A human-readable description of who data marked with the sharing group is
shareable with. This field is NOT used by MISP for anything besides for being an
informational field aimed at extender organisations of the sharing group.
Description: A natural-text representation of the intent of the sharing group.
Make the sharing group selectable (active): A sharing group can be made passive by
unchecking this setting. All events and attributes will continue to adhere to a passive sharing
group's distribution setting, however, the sharing group will not be offered as a selectable
option when setting the distribution of events/attributes. The idea behind this is that ad-hoc
sharing groups that have outlived their purpose can be retired in order to reduce the clutter in
the UI.
Organisations: The second page of the tool contains the distribution list containing all
organisations directly named as a member of the sharing group
Add Local/remote organisations: The organisations are split into two lists (shown as two
tabs in the tool) for local and known remote/external organisations. Local organisations are
expected to have at least one local user on the instance whilst remote organisations do not.
Synchronising with remote instances will create remote organisations whenever a new event
is received of a yet unknown organisation. Remote organisations can always be converted to
local organisations - this becomes interesting if a user of an external organisation requests
access to your MISP instance.
Extend checkmark: Checking the extend checkmark makes the selected organisation an
extender of the sharing group, meaning they can edit the sharing group. It is expected of
these trusted partners that they adhere to the "releasable to" tag set on the general page.
The organisation creating the sharing group is always included as an extender.
Servers: The third page of the tool describes the MISP instances the data marked with the given
sharing group are allowed to be synchronised with. Keep in mind that any user that can view an
event on a given instance will have the right to pull the event to their home instance, as they are
part of the sharing group, however the organisation distribution list will still apply.
Enable roaming mode: This setting will disable the server list and rely purely ont he
organisation list to distribute the data. If a sync connection's host organisation is in the
organisation distribution list the instance becomes eligible for synchronising the data marked
with the sharing group. Generally this carries a slightly higher risk as it relies on
administrators correctly setting up the host organisation settings, but it removes the need to
know the specific instance urls where the event/attribute should flow.
Add instance: Add an instance to the distribution list from the sync instances set up under
sync actions -> servers
All orgs: Checking this checkmark will automatically include all organisations on the given
instance in the sharing group. This means that in order to exchange with all users of a linked
community, one does not need to know every organisation residing on the instance. This
also means that the distribution list will not include the organisation names, which can be
interesting for certain privacy sensitive communities.
Summary: Once everything is set up, MISP will summarise the sharing group in a highlighted
text page, which is highly advised to be reviewed before submiting the new sharing group/editing
the sharing group. Mistakes in the sharing group settings can lead to organisations that should
not be involved in the sharing group getting access or organisations receiving unwanted editing
rights to the sharing group. Keep in mind that even if you have submitted a sharing group, it is not
propagated until an event/attribute receives the sharing group as the selected distribution.
Populate from Template
Templates allow users to rapidly populate events of a specific type by filling out a series of predefined fields. Users with template creation privileges can create new templates for their
organisations or for all organisations on their instance. If you are interested in template creation,
please refer to the templating section. For users trying to populate an event, after clicking on the
populate from template button, you'll be presented with a list of all currently accessible templates. Pick
the one that best describes the event that you are creating.
Once you have chosen a template, you'll be presented with the actual form contained within. Make
sure you fill out as many fields as possible with the mandatory fields - marked by a star in a bracket
such as this: (*) - are filled out. Templates are devided into sections, with each section having a title
and a description in addition to a series of fields. Each field can be an attribute or a file attachment
field. An attribute field has the following components:
Field: The name of the field along with an indication if the field is mandatory.
Description: A short description of the field.
Types: The value(s) that are valid for the field. In the case of several types being shown here, you
can enter value(s) matching any one of the types, or in the case of a batch import field, any
mixture of the given types.
Text field: This field can either be a single line textfield or a multi-line text area. For the former,
enter a single value of the above indicated type, whilst for the latter you cna paste a list of values
separated by line-breaks.
Freetext Import Tool
If you have a list of indicators that you would like to quickly generate attributes out of then the Free-text
import tool is just what you need. Simply paste a list of indicators (separated by line-breaks into this
tool).
Since there are several category / type combinations that can be valid for a lot of values, MISP will
suggest the most common settings. You can alter the category / type / IDS fields manually if you
disagree with the results. The options will be restricted to valid category/type combinations for the
value that you have entered.
If any correlation is already found, these correlations will be displayed in the result page.
Attribute Replace Tool
If you would like to create and maintain an event with a set of indicators that receives removals and
additions over time, then the attribute replace tool might make this task easier for you.
Simply select the desired category / type combination, choose whether the attributes should be
marked for IDS exports and paste the new list of indicators into the textarea. Attributes of the same
category/type that are present in the event but not the new list will be removed, values in the pasted
list that do not yet exist as attributes will be created as attributes and values that already have
matching attributes will be left untouched.
Add attachments to the event:
You can also upload attachments, such as the malware itself, report files from external analysis or
simply artifacts dropped by the malware. Clicking on the add attachment button brings up a form that
allows you to quickly attach a file to the event. The following fields need to be filled out:
Category: The category is the same as with the attributes, it answers the question of what the
uploaded file is meant to describe.
Distribution: This drop-down list allows you to control who will be able to see this attachment.
The distribution is inherited by attributes: the most restrictive setting wins. For more info, refer to
the distribution information in the event section.
Upload field: By hitting browse, you can browse your file system and point the uploader to the file
that you want to attach to the attribute. This will then be uploaded when the upload button is
pushed.
Malware: This check-box marks the file as malware and as such it will be zipped and
passworded, to protect the users of the system from accidentally downloading and executing the
file. Make sure to tick this if you suspect that the filed is infected, before uploading it.
Contextual Comment: You can add some comments to the attribute that will not be used for
correlation but instead serves as purely an informational field.
Propose a change to an event that belongs to another organisation
If you would like to propose a modification to an attribute, or to propose some additional attributes to
the creating organisation, you can do this with the buttons that replace the add attribute field on the
left and the edit icon on the right end of each listed attribute in the event view. The creating
organisation of the event will be able to see any proposals and discard or accept the changes.
If the organisation that has created the event is on another connected server, they will be able to
accept the proposal once they initiate a pull and receive your proposal. After this they can republish
the event, sending the altered attribute back to your instance.
Populate from OpenIOC
It is also possible to attempt to import the data contained in a .ioc file, The import tool will attempt to
gather as many IndicatorItems within nested logical operators as possible without breaking their
validity. After the procedure is done, you'll be presented with a list of successfully created attributes
and a list of failed IndicatorItems as well as a graph of the .ioc file.
Populate from ThreatConnect
You can also import the data from a ThreatConnect export csv file. The following columns are used by
the import tool (and are thus mandatory fields to select during the export):
Type
Value
Confidence
Description
Source
The result will be a list of attributes that get added to the currently selected event, each of which will
be marked with a comment that indicates that its origin being from a ThreatConnect import.
Adding IOCs from a PDF report
You can You can use a generic script called IOC parser or use a script published by Palo Alto to
convert IOC parser output to a MISP event: [report_to_misp] (https://github.com/PaloAltoNetworksBD/report_to_misp/).
Publish an event:
Once all the attributes and attachments that you want to include with the event are uploaded / set, it is
time to finalise its creation by publishing the event (click on publish event in the event view). This will
alert the eligible users of it (based on the private-controls of the event and its attributes/attachments
and whether they have auto-alert turned on), push the event to instances that your instance connects
to and propagate it further based on the distribution rules. It also readies the network related attributes
for NIDS signature creation (through the NIDS signature export feature, for more information, go to the
export section.). There is an alternate way of publishing an event without alerting any other users, by
using the "publish (no email)" button. This should only be used for minor edits (such as correcting a
typo).
If your instance has background jobs enabled then the event might not get published immediately.
Browsing past events:
The MISP interface allows the user to have an overview over or to search for events and attributes of
events that are already stored in the system in various ways.
To list all events:
On the left menu bar, the option "List events" will generate a list of the last 60 events. While the
attributes themselves aren't shown in this view, the following pieces of information can be seen:
Published: Already published events are marked by a checkmark. Unpublished events are
marked by a cross.
Org: The organisation that created the event.
Owner Org: The organisation that owns the event on this instance. This field is only visible to
administrators.
ID: The event's ID number, assigned by the system when the event was first entered (or in the
case of an event that was synchronized, when it was first copied over - more on synchronisation
in chapter xy)
Tags: Tags that are assigned to this event.
#Attr.: The number of attributes that the event has.
Email: The e-mail address of the event's reporter. This is not visible to regular users.
Organisation administrators can see the e-mail addresses of their own organisation's users.
Date: The date of the attack.
Threat Level: The risk level of the attack, the following levels are possible:
Low: General Malware
Medium: Advanced Persistent Threats (APTs)
High: Sophisticated APTs and 0day exploits
Undefined: This field can be left undefined and edited at a later date.
Analysis: Indicates the current stage of the analysis for the event, with the following possible
options:
Initial: The analysis is just beginning
Ongoing: The analysis is in progress
Completed: The analysis is complete
Info: A short description of the event, starting with an internal reference number.
Distribution: This field indicates what the sharing privileges of the event. For details, refer to the
distribution information in the event section.
Actions: The controls that the user has to view or modify the event. The possible actions that are
available (depending on user privileges - click here to find out more about privileges):
Publish: Publishing an event will have several effects: The system will e-mail all eligible
users that have auto-alert turned on (and having the needed privileges for the event,
depending on its private classification) with a description of your newly published event, it
will be flagged as published and it will be pushed to all eligible servers (to read more about
synchronisation between servers, have a look at the section on connecting servers
Edit: Clicking on the edit button will bring up the same same screen as the one used for
creating new events, with the exception that all fields come filled out with the data of the
event that is being edited. The distribution of an event can only be edited if you are a user of
the creating organisation of the event. For more information on this view, refer to the section
on creating an event.
Delete: The system will prompt you before erasing the unwanted event.
View: Will bring up the event view, which besides the basic information contained in the
event list, will also include the following:
Filters
It is also possible to filter the events shown by clicking on the small magnifying glass icons next to the
field names and entering a filter term.
Event view
General Event Information
ID: The ID of the event.
Uuid: In order to avoid collisions between events and attributes (during for example a sync) a
Uuid is assigned that uniquely identifies each of them.
Org The organisation that has originally created the event. The logo (if it exists on the server,
alternatively a string) representing the organisation is also shown int he right upper corner.
Contributors: Shows a list of the organisations that have contributed to the event via proposals. If
you click any of the logos listed here, you'll get redirected to a filtered event history view,
including only the changes made by the organisation.
Tags: A list of tags associated with the event. Clicking a tag will show a list of events with the
same tag attached. The little cross next to each tag allows you to remove the tag from the event,
whilst the '+' button allows you to assign a tag. For the latter two options to be visible, you have to
have tagging permission.
Date: The date of detection, set by the user that creates the event, not to be confused with the
creation date of the event.
Threat Level: The assigned threat level of the event.
Analysis: The status of the analysis.
Distribution: This shows the distribution rules applied to this event, controlling whether only the
creating organisation can see (Your organisation only) it or everyone on the instance (This
community only). The two remaining settings allow the event to be propagated to organisations
on remote connected instances.
Info: A short description of the event itself. Make sure not to put information in here that could be
used for correlation purposes and be better suited as an Attribute.
Published: Whether the event has been published or not. Publishing allows the attributes of the
event to be used for all eligible exports and it notifies users that have subscribed to the event
alerts. Also, a publish initiates a push to all eligible instances.
List of Related Events The list of relations is shown on the right hand side of the general event
information. Events can be related by having one or more attributes that are exact matches. For
example, if two events both contain a source IP attribute of 11.11.11.11 then they are related. The list
of events that are related the currently shown one, are listed under "Related Events", as links (titled
the related event's date and ID number) to the events themselves.
Data Element Toggles You can control some of the data that is shown on this page using three
toggles. The elements that can be disabled are the pivot threads, the attributes (and proposals) and
the Discussions. You can collapse these elements and then expand them again using the same
button.
Pivot Threads While moving from event to event through the relation links (a process that we refer to
as pivoting), you create a path that shows which events you have traversed. This path is reset by
leaving the event view and navigating elsewhere in the application or by deleting the root pivot
element. Each event visited is represented by a bubble in the pivot thread graph, connected by lines
that show how the user has arrived at the next connected event. It is possible to jump back to an
earlier relation and pivot to another event through that, creating branches in the graph. The currently
selected event is coloured blue in the graph. If you would like to delete an element from the graph
(including all of elements that branch off of it) just click on the small x within a pivot bubble. For a
deletion to be possible the following conditions have to be met:
The pivot element to be deleted cannot be on the path that leads to the currently selected event
The pivot element residing in the graph's root can always be deleted - this will simply reset the
current pivot thread
Attributes and Proposals A list of all attributes and proposals attached to the event. The fields for
each of them only differ in the available actions and the fact that for proposals to attributes all fields
are blank that would stay unchanged if the proposal was accepted (for example, proposing a change
to an attribute to turn the IDS flag on will have all fields apart from the IDS flag blank in the proposal.
Here is a list of what each of the fields represents:
Date: The date of the last modification to the attribute. Proposals don't have a date of last edit.
Category: The category of the attribute or proposal. For a list of possible categories visit the
section on categories and types.
Type: The type of the attribute or proposal. For a list of possible categories visit the section on
categories and types.
Value: The value or value-pair of the attribute. This is the main payload of the attribute, which is
described by the category and type columns. For certain types of attributes that are made up of
value-pairs the two parts will be split by a pipe (|), such as for filename|md5. The value field(s) are
used by the correlation engine to find relations between events. In value-pair attributes both
values are correlated individually.
Comment: Attributes can have a contextual comment to further describe the attribute. These
comments are not used for correlation and are purely informative.
Related Events: A list of the event IDs that also contain an attribute with the same value.
IDS: Flags an attribute as an indicator of compromise, allowing it to be included in all of the
eligible exports.
Distribution: Defines the distribution of the attribute individually. An attribute can have a different
distribution level than the event. In any case, the lowest distribution level of the two is used.
Actions: The user can interact with the events through these buttons, which will be further
described in the next portion of the guide as they differ for attributes and proposals.
Depending on the colour coding of the row, you can have an attribute, a proposal to the event or a
proposal to an attribute:
Attributes: Each uncoloured line represents an Attribute.
Proposals to an Event: Each gray line at the end of the list represents a Proposal to an event.
These are proposals for a new attribute, mostly unrelated to any of the currently existing
attributes. If the creator of the event accepts one of these a new attribute will be created.
Proposals to an Attribute: Each attribute can have several edit proposals. These will be placed
right below the attribute that the proposal affects and - as with the event proposals - is coloured
grey. The original attribute's row is coloured blue if a proposal exists for it.
Using the modify button will bring up the attribute creation view, with all data filled out with the
attribute's currently stored data.
Event Discussion Thread
Each event has its own assigned discussion where users (that are eligible to see the event) can
participate in an open discussion. The users are anonymised in the messages, all that other users will
see is their user ID number and their organisation. To post a message on the Event Discussion, either
use the reply button on a previous post or use the quickresponse field at the bottom of the page. Each
post is made up of the following:
Date:The date when the post was created.
Post navigation:This should the post's ID as well as a link to jump to the top of the discussion
thread on the page itself.
Organisation logo:If such an image exists for the organisation that has posted the message, then
the logo is shown.
Message:The body of the post itself. This can also include automatically generated links to other
events and threads as well as show quoted test in embedded bubbles. Editing an event will also
append a post with a message indicating that it was edited together with the timestamp of the
edit.
User:The e-mail address of the poster if he/she is from the organisation as the current user.
Alternatively a generated sting is shown that includes the user ID of the user, so that his/her e-
mail address could remain hidden whilst still being identifiable.
Action buttons:Edit, Delete and Reply. The first two of the three options are only available to the
poster of the message or a site admin. Quoting a post will automatically include the original
message in [quote] tags.
Here is a list of the various tools you can use while using this feature:
Pagination: There are 5 posts visible on each event page, if there have been more messages
posted, use the previous and next button to navigate through the thread. This will not reload the
rest of the page.
Discussion Tags: Users can quote something by encapsulating it in [quote][/quote] tags, they
can create a link to another event with the [event][/event] tags or to another discussion thread with
[thread][/thread].
Quick Post: Adding a post will take the user to a separate add Post page, something that can be
a bit of an inconvenience. To avoid this, there is a quick post button, where users can add
messages on the fly without having to reload the page. On top of the quick post field, 3 buttons
allow users to generate quote, event and thread tags quickly.
Event History:
View the logs of the event that show how the event has changed over time, including the contribution
from other organisations in the form of proposals. There are two ways to get to this view, either by
clicking on View Event History on the side menu of an event view, or by clicking on a contribing
organisation's logo on the event view. The latter will show a restricted form of the logs, showing only
Proposals created by the selected organisation. The fields shown in this view are as described as
follows:
Org: The logo (or in the lack thereof a string representation) of the organisation.
Action: Each entry in the log happens during an action, such as the creation, modification or
deletion of data and some special actions (such as accepting a proposal). This field shows which
action caused the entry to be created.
Model: As described above, a log entry is generated on certain actions. This field shows which
type of data was affected that caused the log entry to be created (such as a change to the event,
the creation of an attribute, the discarding of a proposal, etc).
Title: This is a short description of the change itself and it is not nearly as detailed as the
information administrators get in the audit logs. However, for attributes and proposals the
category / type and value of the created or edited attribute is shown.
Created: The date and time of the log entry's creation.
Listing all attributes:
Apart from having a list of all the events, it is also possible to get a list of all the stored attribute
s in the system by clicking on the list attributes button. The produced list of attributes will include t
he followings fields:
Event: This is the ID number of the event that the attribute is tied to. If an event belongs to your
organisation, then this field will be coloured red.
Org: The organisation that has created the event.
Category: The category of the attribute, showing what the attribute describes (for example the
malware's payload). For more information on categories, go to section xy
Type: The type of the value contained in the attribute (for example a source IP address). For more
information on types, go to section xy
Value: The actual value of the attribute, describing an aspect, defined by the category and type
fields of the malware (for example 11.11.11.11).
Comment: An optional contextual comment attached to the attribute.
IDS: Shows whether the attribute has been flagged for NIDS signature generation or not.
Actions: A set of buttons that allow you to view the event that the attribute is tied to, to edit the
attribute (using the same view as what is used to set up attributes, but filled out with the attribute's
current data) and a delete button.
Searching for attributes:
Apart from being able to list all events, it is also possible to search for data contained in the value field
of an attribute, by clicking on the "Search Attributes" button.
This will bring up a form that lets you enter one or several search strings (separate search strings with
line breaks) that will be compared to the values of all attributes, along with options to narrow down the
search based on category and type. The entered search string has to be an exact match with (the substring of) a value. A second text field makes it possible to enter event IDs for events that should be
excluded from the search (again, each line represents an event ID to be excluded). The third text field
allows the user to restrict the results to attributes from certain organisations or to attributes not created
by certain other organisations, using the above described syntax. The list generated by the search will
look exactly the same as listing all attributes, except that only the attributes that matched the search
criteria will be listed (to find out more about the list attributes view, click here). The search parameters
will be shown above the produced list and the search terms will be highlighted. The last option is a
checkbox that restricts all of the results to attributes that are marked as IDS signatures.
Updating and modifying events and attributes:
Every event and attribute can easily be edited. First of all it is important to find the event or attribute
that is to be edited, using any of the methods mentioned in the section on browsing past events. Once
it is found, the edit button (whether it be under actions when events/attributes get listed or simply on
the event view) will bring up the same screen as what is used to create the entry of the same type (for
an event it would be the event screen as seen here, for an attribute the attribute screen as described
here). Keep in mind that editing any event (either directly or indirectly through an attribute) will
unpublish it, meaning that you'll have to publish it (through the event view) again once you are done.
Tagging:
As described earlier, users with tagging rights can arbitrarily tag events using tags chosen from a pool
of available options. If you have tagging privileges and would like to create a new tag, navigate to
Event Actions - Add Tag. You'll be presented with the following form:
Fill out the following fields:
Name: Pick a name for the tag. Try to use consistent naming conventions across your instance, to
avoid confusion.
Colour: You can choose a colour for the tag by clicking on the colour field and using the colour
picker tool. Try to avoid having duplicate or similar looking colours to help avoid confusion.
Templating:
Newer users can easily be overwhelmed by having to manually populate events with attributes
without any guidance. What sort of information should go into the event? What should be the category
and type of a C2 IP? Templates allow users to use simple forms to populate events. Even though
MISP ships with a few default templates, it is possible for users (with the appropriate templating
privilege) to create new templates for their users or for all users of the instance. Let's look at how you
can create a template. First go to Event Actions - Add Template to go to the event creation view.
The following fields have to be filled out:
Name: The name of the template should describe what type of an event it should be used to
generate attributes.
Tags: You can attach tags to the template - an event populated using the template would
automatically receive the tag(s). Add new tags using the + button. If you chnage your mind about
a tag you can remove it with the cross next to the tag name.
Event Description: A short description about the events that this template should be used for.
Share this template with others: The template can be set to be usable by any organisation on
the instance or only by the one that has created it.
Once the skeleton template is created, you can start populating the template with data. There are 3
types of elements that can be used during the creation of a template: attribute, file and text elements.
Text elements divide the template into sections with an information field, followed by all of the
attribute/file fields until a new text field is read. Don't worry about the order of the elements during
creation, they can be re-arranged using drag & drop. Let's look at the 3 element types:
Attribute Element
The following fields have to be filled out:
Name: The field name that will be presented to the user.
Description: A brief description of the element. Make sure that you provide sufficient information
to the user to make it obvious what is expected.
Category: The category used for any attributes created using this template element.
Type: The type or complex type used for any attributes created using this template element.
Complex types allow for several related types to be used on data entry. For example, a "file"
complex type element allows for filenames and hashes.
Use Complex types: If the category permits it, switch to a complex type using this checkbox.
Automatically mark for IDS: If checked, any attributes generated using this element will be
marked for IDS exporting.
Mandatory element: If the elemnt is marked as mandatory, then the template form can only be
submitted by users if this field is filled out.
Batch import element: Allow for multiple values to be entered (separated by line breaks).
File Element
The following fields have to be filled out:
Name: The field name that will be presented to the user.
Description: A brief description of the element. Make sure that you provide sufficient information
to the user to make it obvious what is expected.
Category: The category to be used by all attachments uploaded through this element.
Malware: If the uploaded files are malicious and should be encrypted and password protected,
mark this checkbox.
Mandatory element: If it should be required to upload an attachment, check this checkbox.
Batch import element: Ticking this checkbox allows users to upload several files using this
element.
Text Element
The following fields have to be filled out:
Name: The name of the section that will be presented to the user.
Text: The description of the section. Explain briefly to the user what the following attribute/file
elements will be dealing with. There are several ways to split a template into sections, try to have
ease of use in mind while creating it.
Contacting the reporter:
To get in touch with the reporter of a previously registered event, just find the event for which you
would like to contact the reporter by either finding it on the list of events, by finding it through one of its
attributes or by finding it through a related event. Once the event is found and the event view opened,
click the button titled "Contact Reporter". This will bring up a view where you can enter your message
that is to be e-mailed to all members of the reporting organisation that subscribe to receiving such
reports or the reporting user himself. Along with your message, the detailed information about the
event in question will be included in the e-mail.
By default, the message will be sent to every member of the organisation that posted the event in the
first place, but if you tick the check-box below the message field before sending the mail, only the
person that reported the event will get e-mailed.
Automation:
It is possible to quickly and conveniently export the data contained within the system using the
automation features located in the main menu on the left (available to users with authentication key
access only). There are various sets of data that can be exported, by using the authentication key
provided by the system (also shown on the export page). If for whatever reason you would need to
invalidate your current key and get a new one instead (for example due to the old one becoming
compromised) just hit the reset link next to the authentication key in the export view or in your "my
profile" view. To find out about the various export formats and the usage within the automation
functions, please read the page on the API's usage.
Exporting data:
For users that do not have authentication key access, an alternate export feature is available that
relies on your interactive login to the site. To access these, just use the export menu button to the left
and you'll be presented with a list of export options. Depending on your server's configuration, you will
be presented with one of two possible pages, depending on whether you have background
processing enabled or not.
Export page with background jobs disabled
The page will list a set of export formats that you can immediately download as a file. Just click on the
desired export format and MISP will start collecting all the data that you will receive in a file. Keep in
mind that this can be a lengthy process. To avoid having to wait, consult with your instance's site
administrator about enabling the background processing.
Export page with background jobs enabled
If the background jobs are enabled, you'll be redirected to a different version of the export page. Here
you will see a table with all of the major export formats and the current status of the cached export
files. Keep in mind that these are generated on an organisation by organisation basis, so even though
others have generated newer export caches your organisation may have an outdated cache. You can
simply issue a generate command (by clicking the "Generate" button) on the desired export type and
the background workers will start fetching and assembling your cache. A progress bar will show the
progress of the export process. Once done, you can click "Download" to download the freshly
generated cache file. If the cache is already up to date from before, then you don't have to regenerate
the cache, just click on the "download" button. You may have noticed that the TEXT export only has a
generate button - this is because TEXT exports are made up of a lot of types of exports, all of which
get generated together. To download any of these files, just click on any of the attribute types at the
bottom of the table. A quick description of each of the fields in the table:
Type: The type of the export (such as XML, Suricata, MD5, etc.).
Last Update: The generation date of the current cache for the given export type.
Description: A description of the export format.
Outdated: This compares the cache generation date to the last timestamp when an event was
updated and lets you know whether the cache is outdated or not.
Progress: Shows the progress of the last initiated generation process.
Actions: Download or Generate the given cache with these buttons.
Exporting search results and individual events
Apart from the options offered by the export pages, it's also possible to export all events involved in a
search attribute result table, by using the "Download results as XML" button on the left menu bar.
Each event's view has its own export feature, both as an XML export and as a .ioc file. To reach these
features, just navigate to an event and use the appropriate buttons on the right side.
Connecting to other instances:
Apart from being a self contained repository of attacks/malware, one of the main features of MISP is its
ability to connect to other instances and share (parts of) its information. The following options allow
you to set up and maintain such connections.
Setting up a connection to another server:
In order to share data with a remote server via pushes and pulls, you need to request a valid
authentication key from the hosting organisation of the remote instance. When clicking on List Servers
and then on New Server, a form comes up that needs to be filled out in order for your instance to
connect to it. The following fields need to be filled out:
Base URL: The URL of the remote server.
Organization: The organisation that runs the remote server. It is very impoportant that this setting
is filled out exactly as the organisation name set up in the bootstrap file of the remote instance.
Authkey: The authentication key that you have received from the hosting organisation of the
remote instance.
Push: This check-box controls whether your server is allowed to push to the remote instance.
Pull: This check-box controls whether your server can request to pull all data from the remote
instance.
Self Signed: Ticking this checkbox will allow syncing with instances using self-signed certificates.
Certificate File: If the instance that you want to connect to has their entire own certificate chain,
you can use this to import a .pem file with it and override CakePHP's standard root CA file.
If you are an administrator, trying to allow another instance to connect to your own, it is vital that two
rules are followed when setting up a synchronisation account:
The synchronisation user has to have the sync permission and full read/write/publish privileges
turned on
Both the sync user and the organisation setting in your instance's Config/bootstrap.php file have
to match the organisation identifier of the hosting organisation.
Browsing the currently set up server connections and interacting
with them:
If you ever need to change the data about the linked servers or remove any connections, you have the
following options to view and manipulate the server connections, when clicking on List Servers: (you
will be able to see a list of all servers that your server connects to, including the base address, the
organisation running the server the last pushed and pulled event IDs and the control buttons.).
Editing the connection to the: By clicking edit a view, that is identical to the new instance view, is
loaded, with all the current information of the instance pre-entered.
Deleting the connection to the instance: Clicking the delete button will delete the link to the
instance.
Push all: By clicking this button, all events that are eligible to be pushed on the instance you are
on will start to be pushed to the remote instance. Events and attributes that exist on the far end
will be updated.
Pull all: By clicking this button, all events that are set to be pull-able or full access on the remote
server will be copied to this instance. Existing events will not be updated.
Rest API:
The platform is also RESTfull, so this means that you can use structured format (XML or JSON) to
access Events data.
Requests
Use any HTTP compliant library to perform requests. You can choose which format you would like to
use as input/output for the REST calls by specifying the Accept and Content-Type headers.
The following headers are required if you wish to recieve / push XML data: Authorization: your
authorisation key Accept: application/xml Content-Type: application/xml
The following headers are required if you wish to recieve / push JSON data: Authorization: your
authorisation key Accept: application/json Content-Type: application/json The following table shows
the relation of the request type and the resulting action:
HTTP format
URL
Controller action invoked
GET
/events
EventsController::index()
GET
/events/123
EventsController::view(123)
POST
/events
EventsController::add()
PUT
/events/123
EventsController::edit(123)
DELETE
/events/123
EventsController::delete(123)
POST
/events/123
EventsController::edit(123)
*Attachments are included using base64 encoding below the
data
tag.
Example - Get single Event
In this example we fetch the details of a single Event (and thus also his Attributes). The request should
be:
GET https://your_misp_url/events/123
And with the HTTP Headers:
Accept: application/xml
The response you're going to get is the following data:
Authorization: your_api_key
<pre><?xml version="1.0" encoding="UTF-8" standalone="no"?>;
<response>
<Event>
<id>57</id>
<org>NCIRC</org>
<date>2014-03-04</date>
<threat_level_id>1</threat_level_id>
<info>Code monkey doing code monkey stuff</info>
<published>1</published>
<uuid>50aa54aa-f7a0-4d74-910d-10f0ff32448e</uuid>
<attribute_count>1</attribute_count>
<analysis>1</analysis>
<timestamp>1393327600</timestamp>
<distribution>1</distribution>
<proposal_email_lock>0</proposal_email_lock>
<orgc>Iglocska</orgc>
<locked>0</locked>
<publish_timestamp>1393327600</publish_timestamp>
<Attribute>
<id>9577</id>
<type>other</type>
<category>Artifacts dropped</category>
<to_ids>1</to_ids>
<uuid>50aa54bd-adec-4544-b494-10f0ff32448e</uuid>
<event_id>57</event_id>
<distribution>1</distribution>
<timestamp>1393327600</timestamp>
<comment>This is an Attribute</comment>
<value>Some_attribute</value>
<ShadowAttribute />
</Attribute>
<ShadowAttribute />
<RelatedEvent />
</Event>
<xml_version>2.2.0</xml_version>
</response>
Example - Add new Event
In this example we want to add a single Event. The request should be:
POST https://your_misp_url/events
Accept: application/xml
Authorization: your_api_key
And the request body:
<Event>
<date>2014-03-04</date>
<threat_level_id>1</threat_level_id>
<info>Something concise</info>
<published>1</published>
<analysis>1</analysis>
<distribution>1</distribution>
<Attribute>
<type>other</type>
<category>Artifacts dropped</category>
<to_ids>1</to_ids>
<distribution>1</distribution>
<comment>This is an Attribute</comment>
<value>Some_attribute</value>
</Attribute>
</Event>
The response you're going to get is the following data:
HTTP/1.1 100 Continue
HTTP/1.1 200 Continue
Date: Tue, 04-Mar-2014 15:00:00
Server: Apache/2.2.22 (Ubuntu) PHP/5.4.9-4ubuntu2.3
X-Powered-By: PHP/5.4.9-4ubuntu2.3
Set-Cookie: CAKEPHP=deleted; expires=Wed, 05-Mar-2014 15:00:00 GMT; path=/
Set-Cookie: CAKEPHP=a4ok3lr5p9n5drqj27025i4le3; expires Tue, 04-Mar-2014 15:00:00 GMT; path=/; HttpOnly
Content-Length: 1 kB
Content-Type: application/xml
<?xml version = "1.0" encoding = "UTF-8"?>
<response>
<Event>
<id>76</id>
<org>NCIRC</org>
<date>2014-03-04</date>
<threat_level_id>1</threat_level_id>
<info>Something concise</info>
<published>1</published>
<uuid>50aa54aa-f7a0-4d74-920d-10f0ff32448e</uuid>
<attribute_count>1</attribute_count>
<analysis>1</analysis>
<timestamp>1393328991</timestamp>
<distribution>1</distribution>
<proposal_email_lock>0</proposal_email_lock>
<orgc>Iglocska</orgc>
<locked>0</locked>
<publish_timestamp>1393947960</publish_timestamp>
<Attribute>
<id>10462</id>
<type>other</type>
<category>Artifacts dropped</category>
<to_ids>1</to_ids>
<uuid>50aa54bd-adec-4544-b412-10f0ff32448e</uuid>
<event_id>76</event_id>
<distribution>1</distribution>
<timestamp>1393328991</timestamp>
<comment/>
<value>Some_attribute</value>
<ShadowAttribute/>
</Attribute>
<ShadowAttribute/>
<RelatedEvent>
<id>75</id>
<org>NCIRC</org>
<date>2012-11-19</date>
<info>Code monkey doing code monkey stuff</info>
<uuid>50aa54aa-f7a0-4d74-910d-10f0ff32448e</uuid>
<published>1</published>
<analysis>1</analysis>
<attribute_count>1</attribute_count>
<orgc>Iglocska</orgc>
<timestamp>1393327600</timestamp>
<distribution>1</distribution>
<proposal_email_lock>0</proposal_email_lock>
<locked>0</locked>
<threat_level_id>1</threat_level_id>
<publish_timestamp>1393947655</publish_timestamp>
</RelatedEvent>
</Event>
<xml_version>2.2.0</xml_version>
</response>
The respone from requesting an invalid page
<?xml version = "1.0" encoding = "UTF-8"?>
<response>
<name>Not Found</name>
<url>/The_meaning_of_life</url>
</response>
Delegation
In information sharing, privacy of the reporting organisation can be important in such case as:
an incident doesn't want to be linked to a potential victim.
to avoid the relation of an organisation with the information shared.
MISP has a functionality to delegate the publication and completely remove the binding between the
information shared and its organisation. If you want to publish an event without you or your
organisation being tied to it, you can delegate the publication to an other organisation. That also
means they will take the ownership of the event.
[warning] You need to have a role with "Delegation access" to delegate an event.
Send a delegation request
To do so, you first need to put the distribution of the event as "your organisation only".
Otherwise the delegation option will not be available.
When the "Delegate Publishing" option is clicked, a pop-up will show up:
Here you can choose
to which organisation you wish to delegate the event among all those registered on the server.
For this example we are going to ask Setec Astronomy to publish the event for us.
The distribution option you would like to put on the event. You can let the other organisation
(called "recipient") choose if you don't mind it. For this example, we will request the recipient to
share it to all communities, but it is only a suggestion, and the recipient will be able to modify the
diffusion setting if wanted.
Finally you can leave a free message to the recipient organisation.
Once the request is sent, a message will appear on the event to remind you of your request.
You can also see more details by clicking on "View request details"
And you can also discard the request your self, by using this pop-up or the link in the left menu.
Answer a delegation request
As the recipient organisation, you will then receive the request of delegation. You will be notified by a
red circle around the envelope on the top right of the screen.
When you click it, you will be redirected as usual on the dashboard, where we can see one
delegation request on the left frame.
Clicking on the "view" link then redirect to an event list view showing all the events other
organisations wish to delegate to your organisation. Here we only see one event, from Acme Factory.
And here are the metadata of the so called event.
You will be able to view the details by clicking the so called link.
If your role have publishing rights, you will be able to manage the delegation request by using one of
the two links in the left menu.
You can either discard it:
Or accept the delegation:
Please notice that the distribution desired by the requester will not automatically be set on the event,
which will stay as distributed to your own organisation only if the parameter is not modified.
Administration
Users
Adding a new user:
Listing all users:
Contacting a user:
Organisations
Adding a new organisation:
Listing all organisation:
Merge organisations:
Roles
Adding a new role:
Listing roles:
Tools
Server Settings
Server settings and diagnostics
Import Blacklist
Adding and modifying entries
Import Regexp
The purpose of Import Regexp entries
Adding and modifying entries
Managing the Signature whitelist
Whitelisting an address:
Managing the list:
Using the logs of MISP
Browsing the logs:
Searching the Logs:
Background Processing
Command Line Tools for the Background Workers
Monitoring the Background Processes
Scheduling Jobs and Recurring Jobs
Various administration tips & tricks
Default sharing level
Adding organisation logos
The _schdlr_ worker is not starting
How to redirect HTTP to HTTPS
Increase max size of Samples / other files
Support & feature requests
More information in the notification emails about new events
Get top API users
MISP Logs
Logging of failed authentication attempts
Clearing expired sessions
Troubleshooting MISP not connecting to redis but redis-cli working
Errors about fields or tables
Administration
Users
Organisations
Roles
Tools
Server Settings
Jobs
Scheduled Tasks
[warning] This page is under modification for updating the content. Current status:
[x] Users
[x] Organisations
[x] Roles
[x] Tools
[ ] Server Settings
[ ] Jobs
[ ] Scheduled Tasks
Users
As an admin, you can set up new accounts for users, edit the profiles of users, delete them, or just
have a look at all the viewers’ profiles. Organisation admins are restricted to executing the same
actions on their organisation’s users only.
Adding a new user:
To add a new user, click on the Add User button in the administration menu to the left and fill out the
following fields in the view that is loaded:
Email: The user's e-mail address, this will be used as his/her login name and as an address to
send all the automatic e-mails and e-mails sent by contacting the user as the reporter of an event.
Set password: Tick the box if you want to define a temporary password for the user. If you don't,
you'll should use the action button 'reset password' on 'List Users' view for generating one and
send it by email to the user.
Password: This textbox is displayed only when 'Set password' is ticked. A Temporary password
for the user that he/she should change after the first login. Make sure that it is at least 6 characters
long, includes a digit or a special character and contains at least one upper-case and at least one
lower-case character.
Confirm Password: This textbox is displayed only when 'Set password' is ticked. This should be
an exact copy of the Password field.
Organisation: A drop-down list allows you to choose an organisation for the user. To learn more
about organisation, click here.
Roles: A drop-down list allows you to choose a role-group that the user should belong to. Roles
define the privileges of the user. To learn more about roles, click here.
Authkey: This is assigned automatically and is the unique authentication key of the user (he/she
will be able to reset this and receive a new key). It is used for exports and for connecting one
server to another, but it requires the user to be assigned to a role that has auth permission
enabled.
NIDS Sid: ID of network intrusion detection systems.
Sync user for: Use this option for granted the user the right to synchronize the event between
MISP server. This option is available for admin, Org Admin and Sync user role.
Gpgkey: The key used for encrypting e-mails sent through the system.
Fetch GPG key: Fetch GPG public key.
Receive alerts when events are published: This option will subscribe the new user to
automatically generated e-mails whenever an event is published.
Receive alerts from "contact reporter" requests: This option will subscribe the new user to emails that are generated when another user tries to get in touch with an event's reporting
organisation that matches that of the new user.
Disable this user account: Tick it if you want to disable this user account.
Listing all users:
To list all current users of the system, just click on List Users under the administration menu to the left.
A view will be loaded with a list of all users and the following columns of information:
Id: The user's automatically assigned ID number.
Org: The organisation that the user belongs to.
Email: The e-mail address (and login name) of the user.
Authkey: Unique authentication key of the user.
Autoalert: Shows whether the user has subscribed to auto-alerts and is always receiving the
mass-emails regarding newly published events that he/she is eligible for.
Contactalert: Shows whether the user has the subscription to contact reporter e-mails directed at
his/her organisation turned on or off.
Gpgkey: Shows whether the user has entered a Gpgkey yet.
Nids Sid: Shows the currently assigned NIDS ID.
Termsaccepted: This flag indicates whether the user has accepted the terms of use or not.
Last login: Date of last login.
Disabled: Show the user status. Enabled or disabled.
Action Buttons: There are 4 options available: reset the password, edit the user, delete the user
or display user's information. These options are also available on the left menu.
Reset Password: Use this action for reseting password. If you have created a new user
without password, tick the 'First time registration' checkbox for sending a welcome message.
Otherwise a reset password message will be sent.
Edit the user: Same options of create user's view. Few options are only available here:
Terms accepted: Indicates whether the user has accepted the terms of use already or
not.
Change Password: Setting this flag will require the user to change password after the
next login.
Reset Auth Key: Use this link for generate a new AuthKey.
Delete the user: If you want to delete a user.
Display the user: Display all user's information.
Contacting a user:
Site admins can use the "Contact users" feature to send all or individual user an e-mail. Users that
have a PGP key set will receive their e-mails encrypted. When clicking this button on the left, you'll be
presented with a form that allows you to specify the type of the e-mail, who it should reach and what
the content is using the following options:
Action: This defines the type of the e-mail, which can be a custom message or a password reset.
Password resets automatically include a new temporary password at the bottom of the message
and will automatically change the user's password accordingly.
Subject: In the case of a custom e-mail, you can enter a subject line here.
Recipient: The recipient toggle lets you contact all your users, a single user (which creates a
second drop-down list with all the e-mail addresses of the users) and potential future users
(which opens up a text field for the e-mail address and a text area field for a PGP public key).
Custom message checkbox: This is available for password resets or for welcome message, you
can either write your own message (which will be appended with a temporary key and the
signature), or let the system generate one automatically.
Keep in mind that all e-mails sent through this system will, in addition to your own message, will be
signed in the name of the instance's host organisation's support team, will include the e-mail address
of the instance's support (if the contact field is set in the bootstrap file), and will include the instance's
PGP signature for users that have a PGP key set (and thus are eligible for an encrypted e-mail).
PGP instance key is the PGP key used by the MISP instance and which is only used to sign
notification. The PGP key used in the MISP instance must not be used anywhere else and should not
be valuable.
Organisations
Each users belongs to an organisation. As admin, you can manage these organisations.
Adding a new organisation:
To add a new organisation, click on the Add Organisation button in the administration menu to the left
and fill out the following fields in the view that is loaded:
Local organisation: If the organisation should have access to this instance, tick this checkbox. If
you would only like to add a known external organisation for inclusion in sharing groups,
uncheck it.
Organisation Identifier: Name your organisation. If you want to add a picture, you should add a
file on the webserver using the 'Server Settings menu'. Picture should have the same name. To
learn more about server settings menu, click here.
Uuid: Unique identifier. If you want to share organisation between MISP multi-instance, use the
same Uuid.
A brief description of the organisation: A word for describing the organisation.
Nationality: A drop-down list for selecting the country of organisation.
Sector: Define the sector of organisation (financial, transport, telecom...)
Type of organisation: Define the type of the organisation.
Contacts: You can add some contact details for the organisation.
Listing all organisation:
To list all current organisation of the system, just click on List Organisations under the administration
menu to the left. There are 3 tabs in this view for filtering the local organisations, remote organisations
and both. Default view display local organisations. For all views the following columns of information
are available:
Id: The organisation's automatically assigned ID number.
Logo: Picture of the organisation.
Name: Name of the organisation.
Uuid: Unique identifier of orgnisation. Share this Uuid for using it between MISP's multi-instance.
Description: Description of the organisation.
Nationality: Country of the organisation.
Sector: Sector defined for the organisation.
Type: Type of organisation.
Contacts: Contacts of orgnisation.
Added by: Login of the user which have added the organisation
Local: Flag defined if the organisation is local or remote.
Actions: There are 3 options available: edit, delete or display organisation's information. These
options are also available on the left menu when you are on the display view.
Edit Organisation: Same options of create organisation's view.
Delete Organisation: Use this option for deleting organisation.
View Organisation: Use this option for displaying information about organisation selected. In
this view, you can display the user belongs to this organisation and events published by
organisation.
Merge organisations:
Merge Organisation menu is available only in the view organisation, under the left menu. Merge one
organisation to another will transfer all users and data from one to another. On the left the
organisation to merge, on the right the target one.
Roles
Privileges are assigned to users by assigning them to rule groups, which use one of four options
determining what they can do with events and four additional privilege elevating settings. The four
options for event manipulation are: Read Only, Manage My Own Events, Manage Organisation
Events, Manage & Publish Organisation Events. A short description is provided below:
Read Only: This allows the user to browse events that his organisation has access to, but doesn't
allow any changes to be made to the database.
Manage My Own Events: The second option, gives its users rights to create, modify or delete
their own events, but they cannot publish them.
Manage Organization Events: Allows users to create events or modify and delete events created
by a member of their organisation.
Manage & Publish Organisation Events: This last setting, gives users the right to do all of the
above and also to publish the events of their organisation.
The extra permissions are defined below:
Perm Admin: Gives the user limited administrator privileges, this setting is used for the
organisation admins.
Perm Audit: Grants access to the logs. With the exception of site admins, only logs generated by
the user's own org are visible.
Perm Tagger: Allow user to assign tags to events.
Perm Sharing Group: Grant access to edit or create sharing groups.
Perm Site Admin: Gives the user full administrator privileges, this setting is used for the site
admins.
Perm Auth: This setting enables the authentication key of the role's users to be used for rest
requests.
Perm Tag Editor: Grand access to edit or create new local tags or from taxonomies.
Perm Delegate: Grant access to delegate the publication of an event to a third-party
organization.
Perm Sync: This setting allows the users of the role to be used as a synchronisation user. The
authentication key of this user can be handed out to the administrator of a remote MISP instance
to allow the synchronisation features to work.
Perm Regexp Access: Allows the users with this permission enabled to edit the regular
expression table. Be careful when giving out this permission, incorrect regular expressions can
be very harmful (infinite loops, loss of data, etc.).
Perm Template: Grant access to create or modify templates.
Adding a new role:
When creating a new role, you will have to enter a name for the role to be created and set up the
permissions (as described above) using the drop-down menu and the check-boxes.
Listing roles:
By clicking on the List Roles button, you can view a list of all the currently registered roles and a list of
the permission flags turned on for each. In addition, you can find buttons that allow you to edit and
delete the roles. Keep in mind that you will need to first remove every member from a role before you
can delete it.
Id: The role's automatically assigned ID number.
Name: The name of role.
Permission: One of the 4 permissions: Read Only, Manage My Own Events, Manage
Organization Events, Manage & Publish Organisation Events.
Extra Permissions flag: Flag for each extra permissions: Admin, Site Admin, Sync Actions, Audit
Actions, Auth key access, Regex Actions, Tagger, Tag Editor, Template Editor, Sharing Group
Editor, Deletagions Access.
Action Buttons: There are 2 options available: Edit Role or Delete it.
Edit Role: Same options of create role's view.
Delete Role: Use this option for deleting a role.
Tools
MISP has a couple of administrative tools that help administrators keep their instance up to date and
healthy. The list of these small tools can change rapidly with each new version, but they should be
self-explanatory. Make sure to check this section after upgrading to a new version, just in case there is
a new upgrade script in there - though if this is the case it will be mentioned in the upgrade
instructions.
Server Settings
Since version 2.3, MISP has a settings and diagnostics tool that allows site-admins to manage and
diagnose their MISP installation. You can access this by navigating to Administration - Server settings.
Server settings and diagnostics
The settings and diagnostics tool is split up into several aspects, all accessible via the tabs ontop of
the tool. For any unset or incorrectly set setting, or failed diagnostic a number next to the tab name will
indicate the number and severity of the issues. If the number is written with a red font, it means that the
issue is critical. First, let's look at the various tabs:
Overview: General overview of the current state of your MISP installation
MISP settings: Basic MISP settings. This includes the way MISP handles the default settings for
distribution settings, whether background jobs are enabled, etc
GnuPG settings: GPG related settings.
Proxy settings: HTTP proxy related settings.
Security settings: Settings controlling the brute-force protection and the application's salt key.
Misc settings: You change the debug options here, but make sure that debug is always disabled
on a production system.
Diagnostics: The diagnostics tool checks if all directories that MISP uses to store data are
writeable by the apache user. Also, the tool checks whether the STIX libraries and GPG are
working as intended.
Workers: Shows the background workers (if enabled) and shows a warning if they are not
running. Admins can also restart the workers here.
Download report: Download a report in JSON format, compiled of all of the settings visible in the
tool.
Each of the setting pages is a table with each row representing a setting. Coloured rows indicate that
the setting is incorrect / not set and the colour determines the severity (red = critical, yellow =
recommended, green = optional). The columns are as follows:
Priority: The severity of the setting.
Setting: The setting name.
Value: The current value of the setting.
Description: A description of what the setting does.
Error Message: If the setting is incorrect / not set, then this field will let the user know what is
wrong.
The workers tab shows a list of the workers that MISP can use. You can restart the workers using the
restart all workers, If the button doesn't work, make sure that the workers were started using the
apache user. This can however only be done using the command line, refer to the INSTALL.txt
documentation on how to let the workers automatically start on each boot.
Worker Type: The worker type is determined by the queue it monitors. MISP currently has 5
queues (cache, default, prio, email and a special schdlr queue).
Worker Id: The ID is made up of the machine name, the PID of the worker and the queue it
monitors.
Status: Displays OK if the worker is running. If the schdlr worker is the only one not running make
sure that you copy the config file into the cakeresque directory as described in the INSTALL.txt
documentation.
Import Blacklist
It is possible to ban certain values from ever being entered into the system via an event info field or an
attribute value. This is done by blacklisting the value in this section.
Adding and modifying entries
Administrators can add, edit or delete blacklisted items by using the appropriate functions in the list's
action menu and the menu on the left.
Import Regexp
The system allows administrators to set up rules for regular expressions that will automatically alter
newly entered or imported events (from GFI Sandbox).
The purpose of Import Regexp entries
They can be used for several things, such as unifying the capitalisation of file paths for more accurate
event correlation or to automatically censor the usernames and use system path variable names
(changing C:\Users\UserName\Appdata\Roaming\file.exe to %APPDATA%\file.exe). The second use
is blocking, if a regular expression is entered with a blank replacement, any event info or attribute
value containing the expression will not be added. Please make sure the entered regexp expression
follows the preg_replace pattern rules as described here
Adding and modifying entries
Administrators can add, edit or delete regular expression rules, which are made up of a regex pattern
that the system searches for and a replacement for the detected pattern.
Managing the Signature whitelist
The signature whitelist view, accessible through the administration menu on the left, allows
administrators to create and maintain a list of addresses that are whitelisted from ever being added to
the NIDS signatures. Addresses listed here will be commented out when exporting the NIDS list.
Whitelisting an address:
While in the whitelist view, click on New Whitelist on the left to bring up the add whitelist view to add a
new address.
Managing the list:
When viewing the list of whitelisted addresses, the following pieces of information are shown: The ID
of the whitelist entry (assigned automatically when a new address is added), the address itself that is
being whitelisted and a set of controls allowing you to delete the entry or edit the address.
Using the logs of MISP
Users with audit permissions are able to browse or search the logs that MISP automatically appends
each time certain actions are taken (actions that modify data or if a user logs in and out). Generally,
the following actions are logged:
User: Creation, deletion, modification, Login / Logout
Event:Creation, deletion, modification, publishing
Attribute: Creation, deletion, modification
ShadowAttribute: Creation, deletion, Accept, Discard
Roles: Creation, deletion, modification
Blacklist: Creation, deletion, modification
Whitelist: Creation, deletion, modification
Regexp: Creation, deletion, modification
Browsing the logs:
Listing all the log entries will show the following columns generated by the users of your organisation
(or all organisations in the case of site admins):
Id: The automatically assigned ID number of the entry.
Email: The e-mail address of the user whose actions triggered the entry.
Org: The organisation of the above mentioned user.
Created: The date and time when the entry originated.
Action: The action's type. This can include: login/logout for users, add, edit, delete for events,
attributes, users and servers.
Title: The title of an event always includes the target type (Event, User, Attribute, Server), the
target's ID and the target's name (for example: e-mail address for users, event description for
events).
Change: This field is only filled out for entries with the action being add or edit. The changes are
detailed in the following format: _variable (initial_value) => (new_value),... When the entry is
about the creation of a new item (such as adding a new event) then the change will look like this
for example: org() => (ADMIN), date() => (20012-10-19),...
Searching the Logs:
Another way to browse the logs is to search it by filtering the results according to the following fields
(the search is a sub-string search, the sub-string has to be an exact match for the entry in the field that
is being searched for):
Email: By searching by Email, it is possible to view the log entries of a single user.
Org: Searching for an organisation allows you to see all actions taken by any member of the
organisation.
Action: With the help of this drop down menu, you can search for various types of actions taken
(such as logins, deletions, etc).
Title: There are several ways in which to use this field, since the title fields contain several bits of
information and the search searches for any substrings contained within the field, it is possible to
just search for the ID number of a logged event, the username / server's name / event's name /
attribute's name of the event target.
Change: With the help of this field, you can search for various specific changes or changes to
certain variables (such as published will find all the log entries where an event has gotten
published, ip-src will find all attributes where a source IP address has been entered / edited, etc).
Background Processing
If enabled, MISP can delegate a lot of the time intensive tasks to the background workers. These will
then be executed in order, allowing the users of the instance to keep using the system without a
hiccup and without having to wait for the process to finish. It also allows for certain tasks to be
scheduled and automated.
Command Line Tools for the Background Workers
The background workers are powered by CakeResque, so all of the CakeResque commands work.
To start all of the workers needed by MISP go to your
/var/www/MISP/app/Console/worker
(assuming a
standard installation path) and execute start.sh. To interact with the workers, here is a list of useful
commands. Go to your
/var/www/MISP/app/Console
(assuming a standard installation path) and execute
one of the following commands as a parameter to
CakeResque.CakeResque tail
./cake CakeResque.CakeResque
(for example:
./cake
):
tail: tail the various log files that CakeResque creates, just choose the one from the list that you
are interested in.
cleanup: terminate the job that a worker is working on immediately. You will be presented with a
choice of workers to choose from when executing this command.
clear: Clear the queue of a worker immediately.
stats: shows some statistics about your workers including the count of successful and failed jobs.
The other commands should not be needed, instead of starting / stopping or restarting workers use
the supplied start.sh (it stops all workers and starts them all up again). For further instructions on how
to use the console commands for the workers, visit the CakeResque list of commands.
Monitoring the Background Processes
The "Jobs" menu item within the Administration menu allows site admins to get an overview of all of
the currently and in the past scheduled jobs. Admins can see the status of each job, and what the
queued job is trying to do. If a job fails, it will try to set an error message here too. The following
columns are shown in the jobs table:
Id: The job's ID (this is the ID of the job's metadata stored in the default datastore, not to be
confused with the process ID stored in the redis database and used by the workers)
Process: The process's ID.
Worker: The name of the worker queue. There are 3+1 workers running if background jobs are
enabled: default, cache, email, and a special Scheduler (this should never show up in the jobs
table).
Job Type: The name of the queued job.
Input: Shows a basic input handled by the job - such as "Event:50" for a publish email alert job for
event 50.
Message: This will show what the job is currently doing or alternatively an error message
describing why a job failed.
Org: The string identifier of the organisation that has scheduled the job.
Status: The status reported by the worker.
Retries: Currently unused, it is planned to introduced automatic delayed retries for the
background processing to add resilience.
Progress: A progress bar showing how the job is coming along.
Scheduling Jobs and Recurring Jobs
Apart from off-loading long-lasting jobs to the background workers, there is a second major benefit of
enabling the background workers: Site-administrators can schedule recurring tasks for the jobs that
generally take the longest to execute. At the moment this includes pushing / pulling other instances
and generating a full export cache for every organisation and export type. MISP comes with these 3
tasks pre-defined, but further tasks are planned. The following fields make up the scheduled tasks
table:
Id: The ID of the task.
Type: The type of the task.
Frequency (h): This number sets how often the job should be executed in hours. Setting this to
168 and picking the next execution on Sunday at 01:00 would execute the task every Sunday at 1
AM. Setting this value to 0 will make the task only run once on the scheduled date / time without
rescheduling it afterwards.
Scheduled Time: The time (in 24h format) when the task should be executed the next time it runs
(and all consecutive times if a multiple of 24 is chosen for frequency).
Next Run: The date on which the task should be executed.
Description: A brief description of the task.
Message: This field shows when the job was queued by the scheduler for execution.
Various administration tips & tricks
Default sharing level
Choose your default sharing level to match your usage scenario for MISP. The setting is named
default_event_distribution and the values can be:
Your organisation only (default)
This community only
Connected communities
All communities
You can also set a default distribution level for attributes contained in an event with
default_attribute_distribution, and it has the same values as the default sharing level for events plus
an additional one that allows attributes to inherit the sharing level of the event.
Adding organisation logos
You can add logo for organisations in MISP by uploading them via the tab Manage files under the
Administration menu & Server Settings sub-menu. The filename must be exactly the same as the
organisation name that you will use in MISP. It is recommended to use PNG files of 48x48 pixels.
The _schdlr_ worker is not starting
If you already made sure that you copied the config file under the cakeresque directory, it might be
due to the FQDN of the server hosting the instance has changed. A way to fix this is to flush temporary
data stored in redis. This can be done by logging in redis, for example when logging in with redis-cli,
and issuing a flushall command.
How to redirect HTTP to HTTPS
Here is a sample configuration for Apache webserver.
<VirtualHost *:80>
ServerAdmin [email protected]
ServerName misp.misp.misp
ServerAlias misp-int.misp.misp
Redirect permanent / https://misp.misp.misp
LogLevel warn
ErrorLog /var/log/apache2/misp.local_error.log
CustomLog /var/log/apache2/misp.local_access.log combined
ServerSignature Off
</VirtualHost>
<VirtualHost *:443>
ServerAdmin [email protected]
ServerName misp.misp.misp
ServerAlias misp-int.misp.misp
DocumentRoot /var/www/MISP/app/webroot
<Directory /var/www/MISP/app/webroot>
Options -Indexes
AllowOverride all
Order allow,deny
allow from all
</Directory>
SSLEngine On
SSLCertificateFile /etc/ssl/misp.misp.misp/misp.crt
SSLCertificateKeyFile /etc/ssl/misp.misp.misp/misp.key
SSLCertificateChainFile /etc/ssl/misp.misp.misp/mispCA.crt
LogLevel warn
ErrorLog /var/log/apache2/misp.local_error.log
CustomLog /var/log/apache2/misp.local_access.log combined
ServerSignature Off
</VirtualHost>
Taken from Koen Van Impe's blog
Increase max size of Samples / other files
Trying to upload a large samples (>50M) might cause the following error:
[!] 500 Server Error:
Internal Server Error
Or will give you an error page in browser.
The error logs on the system will show you the following:
PHP Warning:
POST Content-Length of 57526024 bytes exceeds the limit of 8388608 bytes in Unknown on line
0, referer: https://XYZ/attributes/add_attachment/1948
And / Or
PHP Fatal error:
Allowed memory size of 134217728 bytes exhausted (tried to allocate 76705009 bytes) in
/var/www/MISP/app/Lib/cakephp/lib/Cake/Network/CakeRequest.php on line 996
To fix that you have to adjust the php settings:
vi /etc/php5/apache2/php.ini
Increase to the following values (or more if you like to)
; Maximum size of POST data that PHP will accept.
; Its value may be 0 to disable the limit. It is ignored if POST data reading
; is disabled through enable_post_data_reading.
; http://php.net/post-max-size
post_max_size = 256M
[...]
; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 1024M
And then restart apache2
service apache2 restart
Support & feature requests
The preferred method for support & feature requests is to use the GitHub ticketing system.
If you want to discuss about something related to MISP, want help from the community, etc... You have
the MISP Users mailing list and the MISP developers mailing list.
A number of companies are also offering custom development, consulting, and support around MISP,
please check the support page of the MISP Project website.
More information in the notification emails about new events
The setting MISP.extended_alert_subject allows you to have an extended subject. One word of
warning though. If you’re using encryption : the subject will not be encrypted. Be aware that you might
leak some sensitive information this way. Below is an example how the two subject types look like.
First with the option disabled, then with the option enabled.
Event 7 - Low - TLP Amber
Event 8 - OSINT - Dissecting
XXX... - Low - TLP Amber
Taken from Koen Van Impe's blog
Get top API users
Enable the log_auth setting in the server settings. Optionally enable log_client_ip if you want to get
stats per client ip. Log into your mysql server and run the following query:
select ip,email,count(id) as c from logs WHERE ip IS NOT NULL group by ip,email order by c desc limit 10;
This will give you a top 10 table per ip and username:
+----------------+----------------------------------+------+
| ip
| email
| c
|
+----------------+----------------------------------+------+
| 1.2.3.4
| [email protected]
| 4124 |
| 5.6.7.8
| [email protected]
| 1932 |
| 9.10.11.12
| [email protected]
| 1317 |
| 13.14.15.16
| SYSTEM
|
16 |
+----------------+----------------------------------+------+
MISP Logs
By default, MISP has several layers of logs that can be used to trouble-shoot and to monitor the
system. Let's walk through each of the available logs:
Apache access logs: Rotating logs generated by apache, logging each request, by default (on
Ubuntu) they are found in /var/log/apache2/misp.local_access.log. The location can be changed
via the apache conf file
Apache error logs: Rotating logs generated by apache, logging error messages, by default (on
Ubuntu) they are found in /var/log/apache2/misp.local_error.log. This error log file will generally
not be used by MISP, however, if there is a PHP level error that prevents MISP from functionining
you might have relevant entries here.
MISP error log: Generated by MISP, logging any exceptions that occur during usage. These can
be found in /var/www/MISP/app/tmp/logs/error.log (assuming default installation path). If you are
seeins errors in here and are stuck with an issue let us know via github!
MISP debug log: Generated by MISP, any debug messages and Notice level messages will be
sent to this file. Generally less interesting, but can be helpful during debuging sessions. It should
not be necesary to monitor this under normal usage. The file can be found in
/var/www/MISP/app/tmp/logs/debug.log (assuming default installation path).
MISP worker error log: Generated by MISP background workers, logging any exceptions
generated during a background job. It is the equivalent of the MISP error log for background jobs,
so if scheduled tasks, synchronisation or e-mailing with the workers enabled are causing issues,
this is the place to check. It can normally be found at /var/www/MISP/app/tmp/logs/resque-workererror.log
MISP worker logs: Rotating logs generated by MISP background workers, logging any jobs
executed by workers. This is part of the normal operation of background workers and doesn't
have to be monitored, though it can help when debugging issues. Normally found at
/var/www/MISP/app/tmp/logs/resque-[current date].log
MISP scheduler error log: Generated by MISP scheduler worker, logging any exceptions
generated during the scheduling of a background job. It is the equivalent of the MISP error log for
scheduled jobs. It can normally be found at /var/www/MISP/app/tmp/logs/resque-schedulererror.log
MISP scheduler logs: Rotating logs generated by MISP scheduler worker, logging any
schedulings of jobs to be executed by workers. This is part of the normal operation of the
scheduler worker and doesn't have to be monitored, though it can help when debugging issues.
Normally found at /var/www/MISP/app/tmp/logs/resque-scheduler-[current date].log
Logging of failed authentication attempts
By default, MISP logs all failed login and authentication attempts in the built in Audit logs. To view any
such failed attempts, simply log in as a site admin and navigate to Audit - List logs.
There are two types of entries that will be interesting if you are looking for failed authentication
attempts, entries of action "auth_fail" (for failed attempts to authenticate via the API key or the external
authentication system) and login_fail (for failed login attempts via the login page).
You can also search for any such entries using the Search Logs feature, simply choose the desired
action from the two listed above and hit search.
What is logged:
+----------------+------------+---------------------------+----------+
| Auth method
| Action
| Failed credentials logged | IP
|
+----------------+------------+---------------------------+----------+
| Webform
| login_fail | None
| Optional |
| API
| auth_fail
| API key
| Optional |
| Webform
| auth_fail
| External auth key
| Optional |
+----------------+------------+---------------------------+----------+
In order to enable IP logging for any logged request in MISP, navigate to Administration - Server
settings - MISP settings and enable the MISP.log_client_ip setting.
It is also possible to enable full logging of API and external authentication requests using the
MISP.log_auth setting in the same location, but keep in mind that this is highly verbose and will log
every request made. In addition to the information above, all accessed resource URLs are also
logged.
Clearing expired sessions
By default the garbage collection of sessionsis disabled in PHP. It is possible to enable it, but it's not
recommended and as such MISP provides a manual way of clearing the sessions.
Navigate to the diagnostics screen of MISP (Administration - Server settings - Diagnostics) and near
the bottom of the page there will be a counter showing the count of currently stored expired sessions.
Simply purge them by clicking the applicable button when the number grows too large.
Troubleshooting MISP not connecting to redis but redis-cli working
If you have an IPv6 enabled OS, but an older redis version that does not support IPv6 (<v2.8), MISP
might fail to connect to the redis server while redis-cli is working. The reason is that redis-cli is
connecting to 127.0.0.1 directly, while the calls inside the CakeResque library used by MISP are done
using "localhost" which resolves both to the IPv4 and IPv6 loopback addresses. For some reasons,
the use of the IPv6 address is attempted first which fails.
You can confirm this by trying to connect to redis using telnet localhost 6379. If it fails, the error
message should mention the IPv6 loopback address (::1).
Two ways to fix it:
1) Upgrade your redis to a server that supports IPv6 (v2.8+). This is the preferred recommendation.
2) Comment the localhost mapping to IPv6 address in /etc/hosts
Errors about fields or tables
If you have errors with fields or tables that you can see in the error.log or in the page (if you enabled
debug or site_admin_debug settings), an easy first them to make most of them go away is to use the
clean cache feature on the server settings menu, diagnostics tab. An example of error message:
Error: [PDOException] SQLSTATE[42S22]: Column not found: 1054 Unknown column 'Task.job_id' in 'field list
'
Feeds
Managing feeds
Adding feeds
Feed correlation
Feeds
Feeds are remote or local resources containing indicators that can be automatically imported in MISP
at regular intervals. Feeds can be structured in MISP format, CSV format or even free-text format. You
can easily import any remote or local URL to store them in your MISP instance. It's a simple way to
gather many external sources of information without any programming skills into MISP.
Feeds description can be also easily shared among different MISP instances as you can export a feed
description as JSON and import it back in another MISP instance.
Managing feeds
[warning] A site admin role is required to perform these actions.
To do so, you first need to access the list of feeds, using the top menu.
Adding feeds
Then select the add feed option on the side menu.
Here you will have access to a dynamic form. Let's check each field by order.
Enabled: Is the feed active or not
Lookup Visible: If this is not checked, the correlation will only show up to you, if checked,
correlations are visible for other users as well
Name: Just a name to identify the feed
Provider: Name of the content provider
Input Source: Where does the input come from
Network: hosted somewhere outside the platform
Local: Hosted on the local server. On this case, a new checkbox "Remove input after
ingestion" will appear. If checked, the source is deleted after usage.
Url: Url of the feed, where it is located (for Local hosted files, point to the manifest.json e.g.
/home/user/feed-generator/output/manifest.json)
The Source Format can be:
MISP Feed: The source points to a list of json formated like MISP events.
Example: https://www.circl.lu/doc/misp/feed-osint
Freetext Parsed Feed:
Target Event: Which will be the event getting updated with the data from the feed. Can
be either "New Event Each Pull" (A new event will be created each time the feed is
pulled) or "Fixed Event" (A unique event will be updated with the new data. This event is
determined by the next field)
Target Event ID: The id of the event where the data will be added (if not set, the field will
be set the first time the feed is fetched)
Exclusion Regex: Add a regex pattern for detecting iocs that should be skipped (this can
be useful to exclude any references to the actual report / feed for example)
Auto Publish: If checked, events created thanks to the feed will be automatically
published
Override IDS Flag: If checked, the IDS flag will be set to false
Delta Merge: If checked, only data coming from the last fetch are kept, the old ones are
deleted.
Simple CSV Parsed Feed:
Target Event: Which will be the event getting updated with the data from the feed. Can
be either "New Event Each Pull" (A new event will be created each time the feed is
pulled) or "Fixed Event" (A unique event will be updated with the new data. This event is
determined by the next field)
Target Event ID: The id of the event where the data will be added (if not set, the field will
be set the first time the feed is fetched)
Exclusion Regex: Add a regex pattern for detecting iocs that should be skipped (this can
be useful to exclude any references to the actual report / feed for example)
Value field(s) in the CSV: Select one or several fields that should be parsed by the CSV
parser and converted into MISP attributes
Delimiter: Set the default CSV delimiter (default = ",")
Auto Publish: If checked, events created thanks to the feed will be automatically
published
Override IDS Flag: If checked, the IDS flag will be set to false
Delta Merge: If checked, only data coming from the last fetch are kept, the old ones are
deleted.
Distribution: Define the distribution option that will be set on the event created by the feed
Default Tag: A default tag can be added to the created event(s)
Filter rules: Here you can define which tags or organisations are allowed or blocked.
To add a tag (resp. organisation), first type it into the top middle (resp. bottom middle) text field . Then
use the arrows that point to the outside to add it to the allowed or blocked tags (resp. organisations)
list.
To remove a tag (resp. organisation), select it in the list and click on the arrow pointing to the inside.
Feed correlation
If an indicator from an feed matches an indicator within a MISP event, it will show up as "Feed hits" in
the event overview. The correlation will not show up in the correlation graph of the event.
Automation API
Automation URL
Automation key
Accept and Content-Type headers
XML Export
CSV export
NIDS rules export
Hash - HIDS database export
STIX export
Various ways to narrow down the search results of the STIX export
RPZ export
Text export
RESTful searches with XML result export
Export attributes of event with specified type as XML
Filtering event metadata
Download attachment or malware sample
Download malware sample by hash
Upload malware samples using the "Upload Sample" API
Add or remove tags from events
Proposals API
Sharing groups
Enable and disable feeds via the API
Sightings API
Describe types API
Attribute statistics API
Additional statistics
User management
Organisation management
Discussion API
Automation using PyMISP
Automation API
Automation functionality is designed to automatically generate signatures for intrusion detection
systems. To enable signature generation for a given attribute, Signature field of this attribute must be
set to Yes. Note that not all attribute types are applicable for signature generation, currently we only
support NIDS signature generation for IP, domains, host names, user agents etc., and hash list
generation for MD5/SHA1 values of file artifacts. Support for more attribute types is planned. To to
make this functionality available for automated tools an authentication key is used. This makes it
easier for your tools to access the data without further form-based-authentication.
Automation URL
The documentation will include a default MISP url in the examples. Don't forget to replace it with your
MISP url.
Default MISP url in the documentation:
https://<misp url>/
Automation key
The authentication of the automation is performed via a secure key available in the MISP UI interface.
Make sure you keep that key secret as it gives access to the entire database! The API key is available
in the event actions menu under automation.
Since version 2.2 the usage of the authentication key in the url is deprecated. Instead, pass the auth
key in an Authorization header in the request. The legacy option of having the auth key in the url is
temporarily still supported but not recommended.
The authorization is performed by using the following header:
Authorization: YOUR API KEY
Accept and Content-Type headers
When performing your request, depending on the type of request, you might need to explicitly specify
in what content type you want to get your results. This is done by setting one of the below Accept
headers:
Accept: application/json
Accept: application/xml
When submitting data in a POST, PUT or DELETE operation you also need to specify in what contenttype you encoded the payload. This is done by setting one of the below Content-Type headers:
Content-Type: application/json
Content-Type: application/xml
XML Export
An automatic export of all events and attributes (except file attachments) is available under a custom
XML format.
You can configure your tools to automatically download the following file:
https://<misp url>/events/xml/download
If you only want to fetch a specific event append the eventid number:
https://<misp url>/events/xml/download/1
You can post an XML or JSON object containing additional parameters in the JSON query format or
XML query format. Query parameters provide a way to filter the output to specific parameters.
JSON query format
The URL is appended with json:
https://<misp url>/events/xml/download.json
The query parameters can be the following:
{"request": {"eventid":["!51","!62"],"withAttachment":false,"tags":["APT1","!OSINT"],"from":false,"to":"2
015-02-15"}}
XML query format
The URL is path is:
https://<misp url>/events/xml/download
The query parameters can be the following:
<request><eventid>!51</eventid><eventid>!62</eventid><withAttachment>false</withAttachment><tags>APT1</ta
gs><tags>!OSINT</tags><from>false</from><to>2015-02-15</to></request>
XML download and URL parameters
The XML download also accepts two additional the following optional parameters in the url:
https://<misp url>/events/xml/download/[eventid]/[withattachments]/[tags]/[from]/[to]/[last]
eventid
Restrict the download to a single event
withattachments
A boolean field that determines whether attachments should be encoded and a second parameter
that controls the eligible tags.
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead). For example, to include tag1 and tag2 but exclude tag3
you would use:
https://<misp url>/events/xml/download/false/true/tag1&&tag2&&!tag3
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
The keywords false or null should be used for optional empty parameters in the URL. Also check out
the User Guide to read about the REST API.
CSV export
An automatic export of attributes is available as CSV. Only attributes that are flagged "to_ids" will get
exported.
You can configure your tools to automatically download the following file:
https://<misp url>/events/csv/download
You can specify additional flags for CSV exports as follows:
POST to:
https://<misp url>/events/csv/download
Headers:
Authorization: <your auth key>
Content-type: application/json
Body:
{"parameter1":"value1", "parameter2":1, "parameter3":["value3", "value4", "!value5"]}
eventid
Restrict the download to a single event
ignore
Setting this flag to true will include attributes that are not marked "to_ids".
tags
Simply add a list of tags that should be included or negated (by prepending the tag name with a
"!"). Any event with a negated tag will be ignored, even if an included tag is matching. An example
is included further down.
category
The attribute category, any valid MISP attribute category is accepted.
type
The attribute type, any valid MISP attribute type is accepted.
includeContext
Include the event data with each attribute.
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
For example, to only download a csv generated of the "domain" type and the "Network activity"
category attributes all events except for the one and further restricting it to events that are tagged
"tag1" or "tag2" but not "tag3", only allowing attributes that are IDS flagged use the following syntax:
POST to:
https://<misp url>/events/csv/download
Headers:
Authorization: <your auth key>
Content-type: application/json
Body:
{"tags":["tag1", "tag2", "!tag3"], "category":"Network activity", "type": "domain"}
Alternatively you can fall back to the deprecated syntax of passing parameters in a GET request via
the URL, however this is discouraged:
https://<misp url>/events/csv/download/[eventid]/[ignore]/[tags]/[category]/[type]/[includeContext]/[from
]/[to]/[last]
If you use the deprecated URL parameter method, keep in mind that the keywords false or null should
be used for optional empty parameters. To export the attributes of all events that are of the type
"domain", use the following syntax:
https://<misp url>/events/csv/download/false/false/false/false/domain
NIDS rules export
Automatic export of all network related attributes is available under the Snort or Suricata rule format.
Only published events and attributes marked as IDS Signature are exported.
You can configure your tools to automatically download the following file:
https://<misp url>/events/nids/suricata/download
https://<misp url>/events/nids/snort/download
The full API syntax is as follows:
https://<misp url>/events/nids/[format]/download/[eventid]/[frame]/[tags]/[from]/[to]/[last]
format
The export format, can be "suricata" or "snort"
eventid
Restrict the download to a single event
frame
Some commented out explanation framing the data. The reason to disable this would be if you
would like to concatenate a list of exports from various select events in order to avoid unnecessary
duplication of the comments.
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead). For example, to include tag1 and tag2 but exclude tag3
you would use:
https://<misp url>/events/nids/snort/download/false/false/tag1&&tag2&&!tag3
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 6d or 12h or 30m). This filter will use the published timestamp of the event.
The keywords false or null should be used for optional empty parameters in the URL.
An example for a Suricata export for all events excluding those tagged tag1, without all of the
commented information at the start of the file would look like this:
https://<misp url>/events/nids/suricata/download/null/true/!tag1
Administration is able to maintain a white-list containing host, domain name and IP numbers to
exclude from the NIDS export.
Hash - HIDS database export
Automatic export of MD5/SHA1 checksums contained in file-related attributes. This list can be used to
feed forensic software when searching for suspicious files. Only published events and attributes
marked as IDS Signature are exported.
You can configure your tools to automatically download all the MD5 hashes from MISP:
https://<misp url>/events/hids/md5/download
Or the SHA1 hashes:
https://<misp url>/events/hids/sha1/download
The API's full format is as follow:
https://<misp url>/events/hids/[format]/download/[tags]/[from]/[to]/[last]
format
The export format, can be "md5" or "sha1"
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead). For example, to include tag1 and tag2 but exclude tag3
you would use:
https://<misp url>/events/hids/md5/download/tag1&&tag2&&!tag3
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
The keywords false or null should be used for optional empty parameters in the URL.
For example, to only show sha1 values from events tagged tag1, use:
https://<misp url>/events/hids/sha1/download/tag1
STIX export
You can export MISP events in MITRE's STIX format (to read more about STIX). The STIX XML export
is currently very slow and can lead to timeouts with larger events or collections of events. The STIX
JSON return format does not suffer from this issue.
Usage of the API:
https://<misp url>/events/stix/download
Search parameters can be passed to the function via url parameters or by POSTing an xml or json
object (depending on the return type). The following parameters can be passed to the STIX export
tool: id, withAttachments, tags. Both id and tags can use the && (and) and ! (not) operators to build
queries. Using the url parameters, the syntax is as follows:
https://<misp url>/events/stix/download/[id]/[withAttachments]/[tags]/[from]/[to]/[last]
id
The event's ID
withAttachments
Encode attachments where applicable
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead).
For example, to include tag1 and tag2 but exclude tag3 you would use:
https://<misp url>/events/stix/download/false/true/tag1&&tag2&&!tag3
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
You can post an XML or JSON object containing additional parameters in the following formats.
If you use JSON query objects:
https://<misp url>/events/stix/download.json
{"request": {"id":["!51","!62"],"withAttachment":false,"tags":["APT1","!OSINT"],"from":false,"to":"2015-0
2-15"}}
If you use XML query objects:
https://<misp url>/events/stix/download
<request><id>!51</id><id>!62</id><withAttachment>false</withAttachment><tags>APT1</tags><tags>!OSINT</tag
s><from>false</from><to>2015-02-15</to></request>
Various ways to narrow down the search results of the STIX export
For example, to retrieve all events tagged "APT1" but excluding events tagged "OSINT" and excluding
events #51 and #62 without any attachments:
https://<misp url>/events/stix/download/!51&&!62/false/APT1&&!OSINT/2015-02-15
To export the same events using a POST request use:
https://<misp url>/events/stix/download.json
Together with this JSON object in the POST message:
{"request": {"id":["!51","!62"],"tags":["APT1","!OSINT"],"from":"2015-02-15"}}
XML is automatically assumed when using the STIX export:
https://<misp url>/events/stix/download
The same search could be accomplished using the following POSTed XML object (note that
ampersands need to be escaped, or alternatively separate id and tag elements can be used):
<request><id>!51</id><id>!62</id><tags>APT1</tags><tags>!OSINT</tags><from>2015-02-15</from></request>
RPZ export
You can export RPZ zone files for DNS level firewall by using the RPZ export functionality of MISP.
The file generated will include all of the IDS flagged domain, hostname and IP-src/IP-dst attribute
values that you have access to.
It is possible to further restrict the exported values using the following filters:
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search when passed through the url. Use
semicolons instead (the search will automatically search for colons instead).
id
The event's ID
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-03)
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-03)
MISP will inject header values into the zone file as well as define the action taken for each of the
values that can all be overwritten. By default these values are either the default values shipped with
the application, or ones that are overwritten by your site administrator. The values are as follows:
Value name
RPZ_policy
Default value
DROP
RPZ_walled_garden
127.0.0.1
RPZ_serial
$date00
RPZ_refresh
2h
RPZ_retry
30m
RPZ_expiry
30d
RPZ_minimum_ttl
1h
RPZ_ttl
1w
RPZ_ns
localhost.
RPZ_email
root.localhost
To override the above values, either use the url parameters as described below:
https://<misp url>/attributes/rpz/download/[tags]/[eventId]/[from]/[to]/[policy]/[walled_garden]/[ns]/[em
ail]/[serial]/[refresh]/[retry]/[expiry]/[minim
um_ttl]/[ttl]
Or POST an XML or JSON object with the above listed options:
<request><tags>OSINT&&!OUTDATED</tags><policy>walled-garden</policy><walled_garden>teamliquid.net</walled
_garden><refresh>5h</refresh></request>
{"request": {"tags": ["OSINT", "!OUTDATED"], "policy": "walled-garden", "walled_garden": "teamliquid.net"
, "refresh": "5h"}
Text export
An export of all attributes of a specific type to a plain text file. By default only published and IDS
flagged attributes are exported.
You can configure your tools to automatically download the following files:
https://<misp url>/attributes/text/download/md5
https://<misp url>/attributes/text/download/sha1
https://<misp url>/attributes/text/download/sha256
https://<misp url>/attributes/text/download/filename
https://<misp url>/attributes/text/download/filename|md5
https://<misp url>/attributes/text/download/filename|sha1
https://<misp url>/attributes/text/download/filename|sha256
https://<misp url>/attributes/text/download/ip-src
https://<misp url>/attributes/text/download/ip-dst
https://<misp url>/attributes/text/download/hostname
https://<misp url>/attributes/text/download/domain
https://<misp url>/attributes/text/download/email-src
https://<misp url>/attributes/text/download/email-dst
https://<misp url>/attributes/text/download/email-subject
https://<misp url>/attributes/text/download/email-attachment
https://<misp url>/attributes/text/download/url
https://<misp url>/attributes/text/download/http-method
https://<misp url>/attributes/text/download/user-agent
https://<misp url>/attributes/text/download/regkey
https://<misp url>/attributes/text/download/regkey|value
https://<misp url>/attributes/text/download/AS
https://<misp url>/attributes/text/download/snort
https://<misp url>/attributes/text/download/pattern-in-file
https://<misp url>/attributes/text/download/pattern-in-traffic
https://<misp url>/attributes/text/download/pattern-in-memory
https://<misp url>/attributes/text/download/yara
https://<misp url>/attributes/text/download/vulnerability
https://<misp url>/attributes/text/download/attachment
https://<misp url>/attributes/text/download/malware-sample
https://<misp url>/attributes/text/download/link
https://<misp url>/attributes/text/download/comment
https://<misp url>/attributes/text/download/text
https://<misp url>/attributes/text/download/other
https://<misp url>/attributes/text/download/named pipe
https://<misp url>/attributes/text/download/mutex
https://<misp url>/attributes/text/download/target-user
https://<misp url>/attributes/text/download/target-email
https://<misp url>/attributes/text/download/target-machine
https://<misp url>/attributes/text/download/target-org
https://<misp url>/attributes/text/download/target-location
https://<misp url>/attributes/text/download/target-external
To restrict the results by tags, use the usual syntax. Please be aware the colons (:) cannot be used in
the tag search. Use semicolons instead (the search will automatically search for colons instead). To
get ip-src values from events tagged tag1 but not tag2 use:
https://<misp url>/attributes/text/download/ip-src/tag1&&
It is possible to restrict the text exports on additional flags. The first allows the user to restrict based on
event ID, whilst the second is a boolean switch allowing non IDS flagged attributes to be exported.
Additionally, choosing "all" in the type field will return all eligible attributes.
https://<misp url>/attributes/text/download/[type]/[tags]/[event_id]/[allowNonIDS]/[from]/[to]/[last]
type
The attribute type, any valid MISP attribute type is accepted.
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead).
allowNonIDS
Include attributes that would normally be excluded due to the IDS flag not being set or due to
being whitelisted
from
Set the lowest "date" field value that should be included in the export (format YYYY-MM-DD)
to
Set the highest "date" field value that should be included in the export (format YYYY-MM-DD)
last
Set the timeframe of the export based on the "timestamp" value. The parameter uses a time +
metric notation (valid examples: "2w", "60m", "24h")
For example, to include tag1 and tag2 but exclude tag3 you would use:
https://<misp url>/attributes/text/download/all/tag1&&tag2&&!tag3
event_id
Restrict the results to the given event IDs.
allowNonIDS
Allow attributes to be exported that are not marked as "to_ids".
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
The keywords false or null should be used for optional empty parameters in the URL.
For example, to retrieve all attributes for event #5, including non IDS marked attributes too, use the
following line:
https://<misp url>/attributes/text/download/all/null/5/true
RESTful searches with XML result export
It is possible to search the database for attributes based on a list of criteria.
To return an event with all of its attributes, relations, shadowAttributes, use the following syntax:
https://<misp url>/events/restSearch/download/[value]/[type]/[category]/[org]/[tag]/[quickfilter]/[from]/
[to]/[last]/[eventid]/[withAttachments]/[metadata]/[uuid]
value
Search for the given value in the attributes' value field.
type
The attribute type, any valid MISP attribute type is accepted.
category
The attribute category, any valid MISP attribute category is accepted.
org
Search by the creator organisation by supplying the organisation idenfitier.
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead).
For example, to include tag1 and tag2 but exclude tag3 you would use:
https://<misp url>/events/restSearch/download/null/null/null/null/tag1&&tag2&&!tag3
quickfilter
Enabling this (by passing "1" as the argument) will make the search ignore all of the other
arguments, except for the auth key and value. MISP will return an xml / json (depending on the
header sent) of all events that have a sub-string match on value in the event info, event orgc, or
any of the attribute value1 / value2 fields, or in the attribute comment.
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
eventid
The events that should be included / excluded from the search
withAttachments
Include the attachments/encrypted samples in the export
metadata
Only fetch the event metadata (event data, tags, relations) and skip the attributes
The keywords false or null should be used for optional empty parameters in the URL.
For example, to find any event with the term "red october" mentioned, use the following syntax (the
example is shown as a POST request instead of a GET, which is highly recommended):
POST to:
https://<misp url>/events/restSearch/download
POST message payload (XML):
<request><value>red october</value><searchall>1</searchall><eventid>!15</eventid></request>
POST message payload (JSON):
{"request": {"value":"red october","searchall":1,"eventid":"!15"}}
To just return a list of attributes, use the following syntax:
value
Search for the given value in the attributes' value field.
type
The attribute type, any valid MISP attribute type is accepted.
category
The attribute category, any valid MISP attribute category is accepted.
org
Search by the creator organisation by supplying the organisation identifier.
tags
To include a tag in the results just write its names into this parameter. To exclude a tag prepend it
with a '!'. You can also chain several tag commands together with the '&&' operator. Please be
aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead).
from
Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This
filter will use the date of the event.
to
Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This
filter will use the date of the event.
last
Events published within the last x amount of time, where x can be defined in days, hours, minutes
(for example 5d or 12h or 30m). This filter will use the published timestamp of the event.
eventid
The events that should be included / excluded from the search.
uuid
The returned events must include an attribute with the given UUID, or alternatively the event's
UUID must match the value(s) passed.
The keywords false or null should be used for optional empty parameters in the URL.
https://<misp url>/attributes/restSearch/download/[value]/[type]/[category]/[org]/[tag]/[from]/[to]/[last
]/[eventid]/[withattachments]/[uuid]
Value, type, category and org are optional. It is possible to search for several terms in each category
by joining them with the '&&' operator. It is also possible to negate a term with the '!' operator. Please
be aware the colons (:) cannot be used in the tag search. Use semicolons instead (the search will
automatically search for colons instead). For example, in order to search for all attributes created by
your organisation that contain 192.168 or 127.0 but not 0.1 and are of the type ip-src, excluding the
events that were tagged tag1 use the following syntax:
https://<misp url>/attributes/restSearch/download/192.168&&127.0&&!0.1/ip-src/false/CIRCL/!tag1
You can also use search for IP addresses using CIDR. Make sure that you use '|' (pipe) instead of '/'
(slashes). Please be aware the colons (:) cannot be used in the tag search. Use semicolons instead
(the search will automatically search for colons instead). See below for an example:
https://<misp url>/attributes/restSearch/download/192.168.1.1|16/ip-src/null/CIRCL
Export attributes of event with specified type as XML
If you want to export all attributes of a pre-defined type that belong to an event, use the following
syntax:
https://<misp url>/attributes/returnAttributes/download/[id]/[type]/[sigOnly]
sigOnly is an optional flag that will block all attributes from being exported that don't have the IDS flag
turned on. It is possible to search for several types with the '&&' operator and to exclude values with
the '!' operator. For example, to get all IDS signature attributes of type md5 and sha256, but not
filename|md5 and filename|sha256 from event 25, use the following:
https://<misp url>/attributes/returnAttributes/download/25/md5&&sha256&&!filename/true
Filtering event metadata
As described in the REST section, it is possible to retrieve a list of events along with their metadata by
sending a GET request to the /events API. However, this API in particular is a bit more versatile. You
can pass search parameters along to search among the events on various fields and retrieve a list of
matching events (along with their metadata). Use the following URL:
https://<misp url>/events/index
POST a JSON object with the desired lookup fields and values to receive a JSON back. An example
for a valid lookup:
Authorization: <your API key>
Accept: application/json
Content-type: application/json
Body:
{"searchinfo":"Locky", "searchpublished":1, "searchdistribution":0}
The list of valid parameters:
searchpublished:
Filters on published or unpulished events [0,1] - negatable
searchinfo:
Filters on strings found in the event info - negatable
searchtag:
Filters on attached tag names - negatable
searcheventid:
Filters on specific event IDs - negatable
searchthreatlevel:
Filters on a given event threat level [1,2,3,4] - negatable
searchdistribution:
Filters on the distribution level [0,1,2,3] - negatable
searchanalysis:
Filters on the given analysis phase of the event [0,1,2,3] - negatable
searchattribute:
Filters on a contained attribute value - negatable
searchorg:
Filters on the creator organisation - negatable
searchemail:
Filters on the creator user's email address (admin only) - negatable
searchDatefrom:
Filters on the date, anything newer than the given date in YYYY-MM-DD format is taken - nonnegatable
searchDateuntil:
Filters on the date, anything older than the given date in YYYY-MM-DD format is taken - nonnegatable
Download attachment or malware sample
If you know the attribute ID of a malware-sample or an attachment, you can download it with the
following syntax:
https://<misp url>/attributes/downloadAttachment/download/[Attribute_id]
Download malware sample by hash
You can also download samples by knowing its MD5 hash. Simply pass the hash along as a
JSON/XML object or in the URL (with the URL having overruling the passed objects) to receive a
JSON/XML object back with the zipped sample base64 encoded along with some contextual
information.
You can also use this API to get all samples from events that contain the passed hash. For this
functionality, just pass the "allSamples" flag along. Note that if you are getting all samples from
matching events, you can use all supported hash types (md5, sha1, sha256) for the lookup.
You can also get all the samples from an event with a given event ID, by passing along the eventID
parameter. Make sure that either an event ID or a hash is passed along, otherwise an error message
will be returned. Also, if no hash is set, the allSamples flag will get set automatically.
https:///attributes/downloadSample/[hash]/[allSamples]/[eventID]
POST message payload (XML):
<request><hash>7c12772809c1c0c3deda6103b10fdfa0</hash><allSamples>1</allSamples><eventID>13</eventID</req
uest>
POST message payload (json):
{"request": {"hash": "7c12772809c1c0c3deda6103b10fdfa0", "allSamples": 1, "eventID": 13}}
A description of all the parameters in the passed object:
hash
A hash in MD5 format. If allSamples is set, this can be any one of the following: md5, sha1,
sha256.
allSamples
If set, it will return all samples from events that have a match for the hash provided above.
eventID
If set, it will only fetch data from the given event ID.
Upload malware samples using the "Upload Sample"
API
https://<misp url>/events/upload_sample/[Event_id]
This API will allow you to populate an event that you have modify rights to with malware samples (and
all related hashes). Alternatively, if you do not supply an event ID, it will create a new event for you.
The files have to be base64 encoded and POSTed as explained below. All samples will be zipped
and password protected (with the password being "infected"). The hashes of the original file will be
captured as additional attributes.
The event ID is optional. MISP will accept either a JSON or an XML object posted to the above URL.
The general structure of the expected objects is as follows:
{"request": {"files": [{"filename": filename1, "data": base64encodedfile1}, {"filename": filename2, "data
": base64encodedfile2}],
"optional_parameter1", "optional_parameter2", "optional_parameter3"}}
JSON:
{"request":{"files": [{"filename": "test1.txt", "data": "dGVzdA=="}, {"filename": "test2.txt", "data": "d
GVzdDI="}], "distribution": 1, "info" : "test", "event_id": 15}}
XML:
<request><files><filename>test3.txt</filename><data>dGVzdA==</data></files><files><filename>test4.txt</fi
lename><data>dGVzdDI=</data></files><info>test</info><distribution>1</distribution><event_id>15</event_id
></request>
The following optional parameters are expected:
event_id
The Event's ID is optional. It can be either supplied via the URL or the POSTed object, but the URL
has priority if both are provided. Not supplying an event ID will cause MISP to create a single new
event for all of the POSTed malware samples. You can define the default settings for the event,
otherwise a set of default settings will be used.
distribution
The distribution setting used for the attributes and for the newly created event, if relevant. [0-3]
to_ids
You can flag all attributes created during the transaction to be marked as "to_ids" or not.
category
The category that will be assigned to the uploaded samples. Valid options are: Payload delivery,
Artifacts dropped, Payload Installation, External Analysis.
info
Used to populate the event info field if no event ID supplied. Alternatively, if not set, MISP will
simply generate a message showing that it's a malware sample collection generated on the given
day.
analysis
The analysis level of the newly created event, if applicable. [0-2] threat_level_id: The threat level
ID of the newly created event, if applicatble. [0-3]
comment
This will populate the comment field of any attribute created using this API.
Add or remove tags from events
You can add or remove an existing tag from an event in the following way:
https://<misp url>/events/addTag
https://<misp url>/events/removeTag
Just POST a JSON object in the following format (to the appropriate API depending on whether you
want to add or delete a tag from an event):
{"request": {"Event": {"id": "228", "tag": "8"}}}
Where "tag" is the ID of the tag. You can also use the name of the tag the following way (has to be an
exact match):
{"request": {"Event": {"id": "228", "tag": "OSINT"}}}
Proposals API
You can interact with the proposals via the API directly since version 2.3.148.
HTTP
URL
Explanation
Expected
Payload
Response
/shadow_attributes/view/[proposal_id]
View a
proposal
N/A
ShadowAttribute
object
POST
/shadow_attributes/add/[event_id]
Propose a
new
attribute to
an event
ShadowAttribute
object
ShadowAttribute
object
POST
/shadow_attributes/edit/[attribute_id]
Propose an
edit to an
attribute
ShadowAttribute
object
ShadowAttribute
object
POST
/shadow_attributes/accept/[proposal_id]
Accept a
proposal
N/A
Message
POST
/shadow_attributes/discard/[proposal_id]
Discard a
proposal
N/A
Message
GET
When posting a shadow attribute object, use the following format
JSON:
{"request": {"ShadowAttribute": {"value": "5.5.5.5", "to_ids": false, "type": "ip-dst", "category": "Netw
ork activity"}}}
XML:
<request><ShadowAttribute><value>5.5.5.5</value><to_ids>0</to_ids><type>ip-src</type><category>Network ac
tivity</category></ShadowAttribute></request>
None of the above fields are mandatory, but at least one of them has to be provided.
Sharing groups
MISP allows sharing groups to be retrieved via the API.
https://<misp url>/sharing_groups/index.json
Based on the API key used, the list of visible sharing groups will be returned in a JSON file. The JSON
includes the organization parts of a given sharing group along with the associated server.
Enable and disable feeds via the API
The MISP feeds can be enabled via the API.
A feed can be enabled by POSTing on the following url (feed_id is the id of the feed):
/feeds/enable/feed_id
A feed can be disabled by POSTing on the following url (feed_id is the id of the feed):
/feeds/disable/feed_id
Sightings API
MISP allows Sightings data to be conveyed in several ways.
The most basic way is to POST a blank message to the Sightings API with the attribute ID or attribute
UUID. This will create a sightings entry with the creation of the entry as the timestamp for the
organisation of the authenticated user.
https://<misp url>/sightings/add/[attribute_id]
https://<misp url>/sightings/add/[attribute_uuid]
Alternatively, it is possible to POST a JSON object and gain additional granularity. The following fields
are recognised by the API:
id
The attribute's ID
uuid
The attribute's UUID
value
Will create a sighting for any attribute with the given value or for composite attributes, for the value
matching any element of the attribute value
values
Expects a list, MISP will create sightings for any attribute matching any of the given values or for
composite attributes, for any of the values matching any element of the attribute value
timestamp
Unix timestamp of the sighting, overrides the current time
Some examples:
To create a sighting for attribute #9001:
{"id":"9001"}
To create a sighting for any attribute with the value being teamliquid.net or 173.231.136.216 with the
time of sighting being :
{"values":["teamliquid.net", "173.231.136.216"], "timestamp":1460558710}
It is also possible to POST a STIX indicator with sighting data to the following URL (keep in mind that
the content type has to be XML):
https://<misp url>/sightings/add/stix
MISP will use the sighting's related observables to gather all values and create sightings for each
attribute that matches any of the values. If no related observables are provided in the Sighting object,
then MISP will fall back to the Indicator itself and use its observables' values to create the sightings.
The time of the sighting is the current time, unless the timestamp attribute is set on the Sightings
object, in which case that is taken.
An example STIX sightings document:
<stix:STIX_Package
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:stix="http://stix.mitre.org/stix-1"
xmlns:indicator="http://stix.mitre.org/Indicator-2"
xmlns:stixCommon="http://stix.mitre.org/common-1"
xmlns:cybox="http://cybox.mitre.org/cybox-2"
xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2"
xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1"
xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2"
xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1"
xmlns:example="http://example.com/"
xsi:schemaLocation="
http://stix.mitre.org/stix-1 ../stix_core.xsd
http://stix.mitre.org/Indicator-2 ../indicator.xsd
http://cybox.mitre.org/objects#DomainNameObject-1 http://cybox.mitre.org/XMLSchema/objects/Domain_Nam
e/1.0/Domain_Name_Object.xsd
http://stix.mitre.org/common-1 http://stix.mitre.org/XMLSchema/common/1.1.1/stix_common.xsd
http://cybox.mitre.org/default_vocabularies-2 ../cybox/cybox_default_vocabularies.xsd
http://stix.mitre.org/default_vocabularies-1 ../stix_default_vocabularies.xsd
http://cybox.mitre.org/objects#AddressObject-2 ../cybox/objects/Address_Object.xsd"
id="example:STIXPackage-33fe3b22-0201-47cf-85d0-97c02164528d"
timestamp="2014-05-08T09:00:00.000000Z"
version="1.1.1"
>
<stix:STIX_Header>
<stix:Title>Example watchlist that contains IP information.</stix:Title>
<stix:Package_Intent xsi:type="stixVocabs:PackageIntentVocab-1.0">Indicators - Watchlist</stix:Pa
ckage_Intent>
</stix:STIX_Header>
<stix:Indicators>
<stix:Indicator xsi:type="indicator:IndicatorType" id="example:Indicator-2e20c5b2-56fa-46cd-96628f199c69d2c9" timestamp="2014-05-08T09:00:00.000000Z">
<indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">Domain Watchlist</indicator:Type>
<indicator:Observable id="example:Observable-87c9a5bb-d005-4b3e-8081-99f720fad62b">
<cybox:Object id="example:Object-12c760ba-cd2c-4f5d-a37d-18212eac7928">
<cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN">
<DomainNameObj:Value condition="Equals" apply_condition="ANY">malicious1.example.co
m##comma##malicious2.example.com##comma##malicious3.example.com</DomainNameObj:Value>
</cybox:Properties>
</cybox:Object>
</indicator:Observable>
<indicator:Sightings>
<indicator:Sighting timestamp="2014-05-08T09:00:00.000000Z">
<indicator:Source>
<stixCommon:Identity>
<stixCommon:Name>FooBar Inc.</stixCommon:Name>
</stixCommon:Identity>
</indicator:Source>
<indicator:Related_Observables>
<indicator:Related_Observable>
<stixCommon:Observable id="example:Observable-45b3acdf-1888-4bcc-89a9-6d9f8116fede">
<cybox:Object id="example:Object-a3d36250-42fa-4653-9172-87b87598390c">
<cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN">
<DomainNameObj:Value>malicious2.example.com</DomainNameObj:Value>
</cybox:Properties>
</cybox:Object>
</stixCommon:Observable>
</indicator:Related_Observable>
</indicator:Related_Observables>
</indicator:Sighting>
</indicator:Sightings>
</stix:Indicator>
</stix:Indicators>
</stix:STIX_Package>
POSTing this as the message's body to MISP will sight any attributes visible to the user with he value
"malicious2.example.com". For composite types, a match on a component will also trigger a sighting
(so for example for attributes of type domain|ip a domain match would be sufficient).
If no Related observables are set in the Sighting itself, MISP will fall back to the observable directly
contained in the indicator. So in the following example:
<stix:STIX_Package
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:stix="http://stix.mitre.org/stix-1"
xmlns:indicator="http://stix.mitre.org/Indicator-2"
xmlns:stixCommon="http://stix.mitre.org/common-1"
xmlns:cybox="http://cybox.mitre.org/cybox-2"
xmlns:AddressObject="http://cybox.mitre.org/objects#AddressObject-2"
xmlns:DomainNameObj="http://cybox.mitre.org/objects#DomainNameObject-1"
xmlns:cyboxVocabs="http://cybox.mitre.org/default_vocabularies-2"
xmlns:stixVocabs="http://stix.mitre.org/default_vocabularies-1"
xmlns:example="http://example.com/"
xsi:schemaLocation="
http://stix.mitre.org/stix-1 ../stix_core.xsd
http://stix.mitre.org/Indicator-2 ../indicator.xsd
http://cybox.mitre.org/objects#DomainNameObject-1 http://cybox.mitre.org/XMLSchema/objects/Domain_Nam
e/1.0/Domain_Name_Object.xsd
http://stix.mitre.org/common-1 http://stix.mitre.org/XMLSchema/common/1.1.1/stix_common.xsd
http://cybox.mitre.org/default_vocabularies-2 ../cybox/cybox_default_vocabularies.xsd
http://stix.mitre.org/default_vocabularies-1 ../stix_default_vocabularies.xsd
http://cybox.mitre.org/objects#AddressObject-2 ../cybox/objects/Address_Object.xsd"
id="example:STIXPackage-33fe3b22-0201-47cf-85d0-97c02164528d"
timestamp="2014-05-08T09:00:00.000000Z"
version="1.1.1"
>
<stix:STIX_Header>
<stix:Title>Example watchlist that contains IP information.</stix:Title>
<stix:Package_Intent xsi:type="stixVocabs:PackageIntentVocab-1.0">Indicators - Watchlist</stix:Pa
ckage_Intent>
</stix:STIX_Header>
<stix:Indicators>
<stix:Indicator xsi:type="indicator:IndicatorType" id="example:Indicator-2e20c5b2-56fa-46cd-96628f199c69d2c9" timestamp="2014-05-08T09:00:00.000000Z">
<indicator:Type xsi:type="stixVocabs:IndicatorTypeVocab-1.1">Domain Watchlist</indicator:Type>
<indicator:Observable id="example:Observable-87c9a5bb-d005-4b3e-8081-99f720fad62b">
<cybox:Object id="example:Object-12c760ba-cd2c-4f5d-a37d-18212eac7928">
<cybox:Properties xsi:type="DomainNameObj:DomainNameObjectType" type="FQDN">
<DomainNameObj:Value condition="Equals" apply_condition="ANY">malicious1.example.co
m##comma##malicious2.example.com##comma##malicious3.example.com</DomainNameObj:Value>
</cybox:Properties>
</cybox:Object>
</indicator:Observable>
<indicator:Sightings>
<indicator:Sighting timestamp="2014-05-08T09:00:00.000000Z">
<indicator:Source>
<stixCommon:Identity>
<stixCommon:Name>FooBar Inc.</stixCommon:Name>
</stixCommon:Identity>
</indicator:Source>
</indicator:Sighting>
</indicator:Sightings>
</stix:Indicator>
</stix:Indicators>
</stix:STIX_Package>
MISP would create sightings for attributes matching any of the following: malicious1.example.com,
malicious2.example.com, malicious3.example.com
Describe types API
MISP can procedurally describe all attribute types and attribute categories it currently supports
including the category - type mappings. To access this information simply send a GET request to:
https://<misp url>/attributes/describeTypes
Depending on the headers passed the returned data will be a JSON object or an XML, with 3 main
sections: types, categories, category_type_mappings.
Attribute statistics API
If you are interested in the attribute type or attribute category data distribution on your instance, MISP
offers an API that will create an aggregates list. To access the API, simple sent a GET request to:
https://<misp url>/attributes/attributeStatistics/[context]/[percentage]
Where the following parameters can be set:
Context
Set whether you are interested in the type or category statistics of your instance. This parameter
can be either set to "type" or "category", with type being the default setting if the parameter is not
set.
Percentage
An optional field, if set, it will return the results in percentages instead of the count.
The results are always returned as JSON.
Sample output of the types in percentages from CIRCL's MISP instance:
{
"AS": "0.015%",
"attachment": "0.177%",
"btc": "0.005%",
"campaign-name": "0.005%",
"comment": "1.47%",
"domain": "15.992%",
"domain|ip": "0.005%",
"email-attachment": "0.207%",
"email-dst": "0.121%",
"email-src": "0.192%",
"email-subject": "0.146%",
"filename": "3.698%",
"filename|md5": "0.349%",
"filename|sha1": "0.894%",
"filename|sha256": "0.652%",
"hostname": "17.558%",
"http-method": "0.045%",
"ip-dst": "7.087%",
"ip-src": "2.707%",
"link": "5.748%",
"malware-sample": "0.702%",
"malware-type": "0.005%",
"md5": "21.064%",
"mutex": "0.278%",
"named pipe": "0.03%",
"other": "1.495%",
"pattern-in-file": "0.192%",
"pattern-in-memory": "0.303%",
"pattern-in-traffic": "0.051%",
"regkey": "0.126%",
"regkey|value": "0.187%",
"sha1": "8.921%",
"sha256": "5.597%",
"snort": "0.045%",
"target-machine": "0.248%",
"target-org": "0.01%",
"target-user": "0.106%",
"text": "0.934%",
"threat-actor": "0.005%",
"url": "2.258%",
"user-agent": "0.081%",
"vulnerability": "0.182%",
"whois-registrant-email": "0.01%",
"x509-fingerprint-sha1": "0.01%",
"yara": "0.086%"
}
Additional statistics
Additional statistics are available as JSON which are the statistics also usable via the user interface. A
".json" can be appended to the following URLs:
https:///users/statistics/tags.json
https:///users/statistics.json
https:///users/statistics/attributehistogram.json
https:///users/statistics/orgs.json
An example output of https:///users/statistics.json:
{
"stats": {
"event_count": 5233,
"event_count_month": 21,
"attribute_count": 645498,
"attribute_count_month": 723,
"correlation_count": 207152,
"proposal_count": 48944,
"user_count": 1073,
"org_count": 587,
"thread_count": 191,
"thread_count_month": 0,
"post_count": 337,
"post_count_month": 0
}
}
User management
MISP allows administrators to create and manage users via its REST API
The API is available in JSON format so make sure you use the following headers:
Authorization: [Your auth key]
Content-type: application/json
Accept: application/json
To fetch all users send a GET request to:
https://<misp url>/admin/users
To view a user simply send a GET request to the following url:
https://<misp url>/admin/users/view/[user id]
To create a new user, send a POST request to:
https://<misp url>/admin/users/add
Sample input:
{
"email":"[email protected]",
"org\_id":1,
"role\_id":1
}
To view the mandatory and optional fields, use a GET request on the above URL.
Sample output:
{
"name": "\/admin\/users\/add API description",
"description": "POST a User object in JSON format to this API to create a new user.",
"mandatory_fields": [
"email",
"org_id",
"role_id"
],
"optional_fields": [
"password",
"external_auth_required",
"external_auth_key",
"enable_password",
"nids_sid",
"server_id",
"gpgkey",
"certif_public",
"autoalert",
"contactalert",
"disabled",
"change_pw",
"termsaccepted",
"newsread"
],
"url": "\/admin\/users\/add"
}
To edit an existing user send a POST request to:
https://<misp url>/admin/users/edit/[user id]
Only the fields POSTed will be updated, the rest is left intact. To view all possible parameters, simply
send a GET request to the above URL.
You can also delete users by POSTing to the below URL, but keep in mind that disabling users (by
setting the disabled flag via an edit) is always prefered to keep user associations to events intact.
https://<misp url>/admin/users/delete/[user id]
Organisation management
MISP allows administrators to create and manage organisations via its REST API
The API is available in JSON format so make sure you use the following headers:
Authorization: [Your auth key]
Content-type: application/json
Accept: application/json
To fetch all organisations send a GET request to:
https://<misp url>/organisations
To view an individual organisation, send a get request to:
https://<misp url>/organisations/view/id
The management of users happens via three apis:
https://<misp url>/admin/organisations/add
https://<misp url>/admin/organisations/edit/[org id]
https://<misp url>/admin/organisations/delete/[org id]
To delete an organisation simply send a POST or DELETE request to the above URL.
For creating or modifying an organisation, simply POST a JSON containing the relevant fields to the
appropriate API. The only mandatory field is the organisation name, with a host of optional
parameters
An example for a simple organisation object:
{
"name": "Blizzard",
"nationality": "US"
}
Not setting a field will assume the default settings for the given field in the case of a new organisation
whilst it would retain the existing setting for existing organisations. The above example would create
the following object in MISP:
{
"Organisation": {
"id": "1108",
"name": "Blizzard",
"alias": "",
"anonymise": false,
"date_created": "2017-01-22 17:32:29",
"date_modified": "2017-01-22 17:32:29",
"description": "",
"type": "",
"nationality": "US",
"sector": "",
"created_by": "1",
"uuid": "5884de9d-04f0-4d7d-bf15-0b88c0a83865",
"contacts": "",
"local": true,
"landingpage": ""
}
}
To query the add or edit APIs for the valid parameters, simply send a GET requests to either API. The
result currently looks like this (which might change when new fields are added):
{
"name": "\/admin\/organisations\/add API description",
"description": "POST an Organisation object in JSON format to this API to create a new organsiation."
,
"mandatory_fields": [
"name"
],
"optional_fields": [
"anonymise",
"description",
"type",
"nationality",
"sector",
"uuid",
"contacts",
"local"
],
"url": "\/admin\/organisations\/add"
}
Discussion API
If you would like to fetch a discussion thread including all of its posts, simply send a GET request to:
https://<misp url>/threads/view/<thread id>
Using the following headers:
Authorization: [Your auth key]
Content-type: application/json
Accept: application/json
To get all posts related to an event simply send a GET request to:
https://<misp url>/threads/viewEvent/<event id>
Automation using PyMISP
PyMISP is a Python library to access MISP platforms via their REST API.
PyMISP allows you to fetch events, add or update events/attributes, add or update samples or search
for attributes.
PyMISP is available including a documentation with various examples.
PyMISP - Python Library to access MISP
PyMISP is a Python library to access MISP platforms via their REST API.
PyMISP allows you to fetch events, add or update events/attributes, add or update samples or search
for attributes.
Note that you need to have Auth Key access in your MISP instance to use PyMISP
Capabilities
Add, get, update, publish, delete events
Add or remove tags
Add file attributes: hashes, registry key, patterns, pipe, mutex
Add network attributes: IP dest/src, hostname, domain, url, UA, ...
Add Email attributes: source, destination, subject, attachment, ...
Upload/download samples
Update sightings
Proposals: add, edit, accept, discard
Full text search and search by attributes
Get STIX event
Export statistics
And even more, just look at the api.py file
Installation
You can install PyMISP by either using pip or by getting the last version from the GitHub repository
Install from pip
pip install pymisp
Install the lastest version from repo
git clone https://github.com/MISP/PyMISP.git && cd PyMISP
python setup.py install
Note that you will also need to install requests if you don't have it already.
Getting started
You now need to get your automation key. You can find it on the automation page:
https://<misp url>/events/automation
or on your profile
https://<misp url>/users/view/me
If you did not install using the repository, you can still fetch it to get examples to work on:
git clone https://github.com/MISP/PyMISP.git
In order to use these, you need to create a file named keys.py in the examples folder and edit it to put
the url of your MISP instance and your automation key.
cd examples
cp keys.py.sample keys.py
vim keys.py
Once you are done with it, you are ready to start.
Using PyMISP
To have a better understanding of how to use PyMISP, we will have a look at one of the existing
examples: add_named_attribute.py This script allow us to add an attribute to an existing event while
knowing only its type (the category is determined by default).
#!/usr/bin/env python
# -*- coding: utf-8 -*from pymisp import PyMISP
from keys import misp_url, misp_key
import argparse
First of all, it is obvious that we need to import PyMISP. Then we also need to know both the instance
with which we will work and the API key to use: Both should be stored in the keys.py file.
Finally we import argparse library so the script can handle arguments.
# For python2 & 3 compat, a bit dirty, but it seems to be the least bad one
try:
input = raw_input
except NameError:
pass
Just a few lines to be sure that pyhon 2 and 3 are supported
def init(url, key):
return PyMISP(url, key, True, 'json', debug=True)
This function will create a PyMISP object that will be used later to interact with the MISP instance. As
seen in the api.py, a PyMISP object need to know both the url of the MISP instance and the API key to
use. It can also take additionnal and not mandatory data, such as the use or not of SSL or the name of
the export format.
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Create an event on MISP.')
parser.add_argument("-e", "--event", type=int, help="The id of the event to update.")
parser.add_argument("-t", "--type", help="The type of the added attribute")
parser.add_argument("-v", "--value", help="The value of the attribute")
args = parser.parse_args()
Then the function starts by preparing the awaited arguments:
event: The event that will get a new attribute
type: The type of the attribute that will be added. See here for more informations
value: The value of the new attribute
misp = init(misp_url, misp_key)
Thanks to the previously created function, we create a PyMISP object.
event = misp.get_event(args.event)
event = misp.add_named_attribute(event, args.type, args.value)
In order to add the new argument, we first need to fetch the event in the MISP database using the
get_event function which only need the event_id. Then only once we have it, we can call the
function add_named_attribute that will add the argument.
print(event)
Finally the new event is printed, so we can check that the attribute was correctly added, and that a
category was attached to it automatically.
Existing examples
As the name implies you will find several example scripts in the examples folder. For each you can
get help if you do
scriptname.py -h
Let us have a look at some of these examples:
add_named_attribute.py
Allow to add an argument to an existing event by giving only the type of the attribute. The category will
be set with a default value.
Arguments:
event: The id of the event to update.
type: The type of the added attribute.
value: The value of the attribute.
add_user.py
Allow to add a user by giving the mandatory fields as entries.
Arguments:
email: Email linked to the account.
org_id: Organisation linked to the user.
role_id: Role linked to the user.
add_user_json.py
Add the user described in the given json. If no file is provided, returns a json listing all the fields used
to describe a user.
Arguments:
json_file: The name of the json file describing the user you want to create.
create_events.py
Allow a user to create a new event on the MISP instance.
Arguments:
distrib: The distribution setting used for the attributes and for the newly created event, if relevant.
[0-3].
info: Used to populate the event info field if no event ID supplied.
analysis: The analysis level of the newly created event, if applicable. [0-2]
threat: The threat level ID of the newly created event, if applicable. [1-4]
del.py
Delete an event or an attribute from a MISP instance. The event has the priority: if both are set, only
the event will be deleted.
Arguments:
event: Event ID to delete.
attribute: Attribute ID to delete.
delete_user.py
Delete the user with the given id. Keep in mind that disabling users (by setting the disabled flag via an
edit) is always prefered to keep user associations to events intact.
Arguments:
user_id: The id of the user you want to delete.
edit_user.py
Edit the email of the user designed by the user_id.
Arguments:
user_id: The name of the json file describing the user you want to modify.
email: Email linked to the account.
edit_user_json.py
Edit the user designed by the user_id. If no file is provided, returns a json listing all the fields used to
describe a user.
Arguments:
user_id: The name of the json file describing the user you want to modify.
json_file: The name of the json file describing your modifications.
get.py
Get an event from a MISP instance in json format.
Arguments:
event: Event ID to get.
output: Output file
last.py
Download latest events from a MISP instance. A output file can be created to store these events.
Arguments:
last: can be defined in days, hours, minutes (for example 5d or 12h or 30m).
output: Output file
searchall.py
Get all the events matching a value.
Arguments:
search: String to search.
quiet: Only display URLs to MISP
output: Output file
sharing_groups.py
Get a list of the sharing groups from the MISP instance.
No argument.
sighting.py
Add sighting.
Arguments:
json_file: The name of the json file describing the attribute you want to add sighting to.
stats.py
Output attributes statistics from a MISP instance.
No argument.
suricata.py
Download Suricata events.
Arguments:
all: Download all suricata rules available.
event: Download suricata rules from one event.
tags.py
Get tags from MISP instance.
No argument.
tagstatistics.py
Get statistics from tags.
Arguments:
percentage: An optional field, if set, it will return the results in percentages, otherwise it returns
exact count.
namesort: An optional field, if set, values are sort by the namespace, otherwise the sorting will
happen on the value.
up.py
Update an existing event regarding the data inside a given json file.
Arguments:
event: Event ID to modify.
input: Input file
upload.py
Send malware sample to MISP.
Arguments:
upload: File or directory of files to upload.
event: Not supplying an event ID will cause MISP to create a single new event for all of the
POSTed malware samples.
distrib: The distribution setting used for the attributes and for the newly created event, if relevant.
[0-3].
ids: You can flag all attributes created during the transaction to be marked as \"to_ids\" or not.
categ: The category that will be assigned to the uploaded samples. Valid options are: Payload
delivery, Artifacts dropped, Payload Installation, External Analysis.
info: Used to populate the event info field if no event ID supplied.
analysis: The analysis level of the newly created event, if applicatble. [0-2]
threat: The threat level ID of the newly created event, if applicatble. [1-4]
comment: Comment for the uploaded file(s).
users_list.py
Get a list of the sharing groups from the MISP instance.
No argument.
Going further
feed-generator
It is used to generate the CIRCL OSINT feed. This script export the events as json, based on tags,
organisation, events, ... It automatically update the dumps and the metadata file.
Here is an example of a config file:
url = ''
key = ''
ssl = True
outputdir = 'output'
# filters = {'tag' : 'tlp : white|feed-export|!privint', 'org':'CIRCL'}
filters = {}
valid_attribute_distribution_levels = ['0', '1', '2', '3', '4', '5']
Consuming feed
As the feed is a simple set of MISP json files, the files can be easily imported directly into any MISP
instance. The script below processes the manifest file of an OSINT feed and reimport them in a MISP
directly.
#!/usr/bin/env python
# -*- coding: utf-8 -*from pymisp import PyMISP
import requests
url = 'https://www.circl.lu/doc/misp/feed-osint/'
osintcircl = requests.get('{}manifest.json'.format(url))
misp = PyMISP('http://misp.test/', 'key', False, 'json')
for uri in osintcircl.json():
req = requests.get('{}{}.json'.format(url,uri))
misp.add_event(req.json())
ioc-2-misp
Allow to import OpenIOC files into MISP easily. It is also possible to set specific tags on these events.
Situational Awareness
attribute_treemap.py generate a treemap showing the distribution of the attributes on the misp
instance.
tags_* : these functions help having statistics and graphs about the tag repartition.
Create an event based on a report
[warning] A specific permission is required to create an event.
For this example, we will use a report found on Bleeping Computer, so considered as OSINT.
The metadata
First of all, we need to create a new event. To do so, we click the "Add Event" option when on the
Events list view.
Then we get the add event form.
Let's fill it with the data we already have:
Date: Here we will put the date of the report, so 2016-11-14
Distribution: Depending on the event, we might want it to be more or less spread accross the
MISP instances. For this one, since it is a public report, there is no reason to limit the diffusion so
"All communities".
Threat Level: Self explainatory. Since the ransomware in the report is not using a huge exploit,
we can use low, or undefined as we don't really know. we'll go for the latter since it can be edited.
Analysis: Give the current stage of the analysis. Since the report is published, we can assume
that the analysis is completed.
Event Info: The event's info is in fact the name or title of the event, so it seems legit to put the title
of the report here as well. Since it is public information, we also prefix it with "OSINT".
GFI sandbox: Since we don't have any sample or anything here, we leave this alone.
Then just press the blue "Add" button and here we have a brand new event. Empty.
(Displayed information can change depending on your role on the MISP instance)
Now it is time to populate this event. But before even adding IoC, we are going to add global
information about the report itself: the link of the report and a short explanation or introduction. To do
so, we need to click on the "Add Attribute" option in the side menu. This will show us this view:
First we are going to add the link of the report. Since it has been written by an other researcher, it
will be considered as an "External analysis", we choose this category.
Concerning the type, regarding the kind of data we are adding it is obvious that we will choose
the "link" type.
The distribution field can be a little tricky. We can either choose one of the option that was already
available at event level or "Inherit event". If we choose the latter, the attribute will be shared the
same way as the event it is included in (here to "All communities"). On the other hand, if we
choose manually a distribution for the attribute, the most restritive between event distribution and
attribute distribution will be applied. That is to say: if both event and attribute distributions are the
same, there will be no change (similar to "Inherit event"). However, if for instance the event
distribution is "all communities" while the attribute is limited to "This community only", the event
will indeed be distributed to all communities but without this particular attribute which will be
limited to this community only. The same works the other way around, if the attribute can be
distributed to "all communities" while the related event is limited to this community, the attribute
being dependant of the event, it will be shared to this community only, basing its distribution on
the event (most restrictive) one.
The value is simply the data we want to add, here it is the link of the report.
The contextual comment is a field that will not be used for correlation and is mainly there to add
some complementary information on the attribute. Can be a port for an IP, or an indication of any
type. Here there is no perticular information to add, except maybe tell that it is the source of the
report, so let us put this information.
"for Intrusion Detection System" is used to set the IDS flag or not. If set, the attribute will be used
as an IDS signature when exporting the NIDS data. In this case, we have no reason to check it.
The Batch Import is a useful option when we need to add several IoC of the same category/type
which allow you to add them at once by separated by a line break between each line in the value
field. However it is of no use here.
All fields are properly filled ? Then let's press the "submit" button, and Ta-dah !
Now we can do a similar procedure to add an introduction to the report (that is to say the first
paragraph of the report). We will simply change the type for text. But this time, we will access the add
attribute form by clicking on the small + symbol next to the attribute table.
The same form as before will appear in a popup.
Again, we fill it with the required data.
Then we submit it by clicking on the blue button Et voilà!
Okay, now it is time to add some Indicators of Compromise. In this report, they are mainly listed at the
end.
Let's try to define which category/type those IoC belong to.
First, Windows-TuneUp.exe is without a doubt a filename, and the associated category may be
Payload delivery.
Second the registry entries (type regkey) seems to be from Artifacts dropped category
Then the hashes that are already said to be SHA 256, and a quick test on VirusTotal also reveals that
they correspond to the filename seen earlier. so we can add both as an association
filename|SHA256. Once again, the category will be Payload delivery.
And finally the network communication. No doubt here for the category: Network activity, and the type
might be url but for the example, we will let MISP decide for us.
So we begin with the filename. No real change from before for this one, except that we will set the IDS
flag to true.
Then we can add the hashes in a similar way. We will had them both alone and combined with the
filename. In order to do it quickly, we are going to use the freetext import tool, hidden there
It will open a popup with a text area field where we will paste our IoC, one per line. As said previously,
we add both the hashes alone and with the filename.
Then when we press the submit button, we are redirected on this page to control the sent data.
Here, MISP detected by itself what should be the category and type associated to our IoC and
surprise! It matches our suppositions. Plus, it also put the IDS flag, so it is perfect. But before
submitting, please double check to be sure all the values are correct and no information was lost
(That can happen when the data are not formatted as expected by MISP).
If the results of MISP were not what we expected, we can still modify it, however MISP will only
suggest suitable category/type regarding the format of your data. We can change for each attribute
individually or all at the same time using the option on the bottom right of the form. The same principle
also applies for the comments, individually or for all.
(Yes I have two cursors, MISP is magic!)
We only have the network indicators left, and as said before, we will let MISP determined for us which
type is the best for the data we have.
Oh well, that was unexpected. In fact, it is not that surprising regarding the format of the tor address
that look more like a filename than like a url but it is still a problem, since we can't change the type nor
the category to a more consistant one. This is indeed one of the limitation of freetext import. To solve
this issue, we will use a simple trick: we will add a slash at the end of the tor address so it won't be
confused for a filename.
Thanks to the added character, the first string is recognised as an url which is more consistent with the
reality. The second also seems okay, so we can now submit both.
And that is all we can get for the main informations and IoC in this report. If we search more carefully,
there might still be some information left in it, like the filename of the ransomnote for instance, but we
will stop here for this example.
Taxonomies
Contributing to Taxonomy
Reserved Taxonomy
Adding Taxonomy in MISP
Adding a private taxonomy
How using Taxonomy in MISP
Filtering the distribution of events among MISP instances
MISP Taxonomies - tools
Other use cases using MISP taxonomies
MISP warning lists: The dilemma of false-positive
Future functionalities related to MISP taxonomies
Taxonomies
In MISP 2.4, a flexible mechanism has been introduced to support various taxonomy of classification.
You can access the taxonomy by going into 'Event Actions' and select 'List Taxonomies'.
The following taxonomies can be used in MISP (as local or distributed tags) or in other tools willing to
share common taxonomies among security information sharing tools.
The following taxonomies are described:
1. Admiralty Scale: The Admiralty Scale (also called the NATO System) is used to rank the reliability
of a source and the credibility of an information.
2. adversary An overview and description of the adversary infrastructure.
3. CIRCL Taxonomy - Schemes of Classification in Incident Response and Detection CIRCL
Taxonomy is a simple scheme for incident classification and area topic where the incident took
place.
4. Cyber Kill Chain from Lockheed Martin as described in Intelligence-Driven Computer Network
Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains.
5. DE German (DE) Government classification markings (VS) Taxonomy for the handling of
protectively marked information in MISP with German (DE) Government classification markings
(VS).
6. DHS CIIP Sectors DHS critical sectors as described in https://www.dhs.gov/critical-infrastructuresectors.
7. Diamond Model for Intrusion Analysis, a phase-based model developed by Lockheed Martin,
aims to help categorise and identify the stage of an attack.
8. Domain Name Abuse - taxonomy to tag domain names used for cybercrime. Use europolincident to tag abuse-activity
9. eCSIRT eCSIRT incident classification Appendix C of the eCSIRT EU project including IntelMQ
updates.
10. ENISA ENISA Threat Taxonomy - A tool for structuring threat information as published
11. Estimative Language Estimative language - including likelihood or probability of event based on
the Intelligence Community Directive 203 (ICD 203) (6.2.(a)).
12. [EU Marketop and Publicadmin]EU critical sectors Market operators and public administrations
that must comply to some notifications requirements under EU NIS directive.
13. EUCI EU classified information (EUCI) means any information or material designated by a EU
security classification, the unauthorised disclosure of which could cause varying degrees of
prejudice to the interests of the European Union or of one or more of the Member States as
described.
14. Europol Incident EUROPOL class of incident taxonomy
15. Europol Events - EUROPOL type of events taxonomy
16. FIRST CSIRT Case FIRST CSIRT Case Classification.
17. FIRST Information Exchange Policy (IEP) framework
18. French gov information classification system
19. Information Security Indicators Information security indicators have been standardized by the
ETSI Industrial Specification Group (ISG) ISI. These indicators provide the basis to switch from a
qualitative to a quantitative culture in IT Security Scope of measurements: External and internal
threats (attempt and success), user's deviant behaviours, nonconformities and/or vulnerabilities
(software, configuration, behavioural, general security framework).
20. Information Security Marking Metadata (ISM) V13 as described by DNI.gov.
21. Malware classification based on different categories. Based on a SANS whitepaper about
malware.
22. Malware Type and Platform classification based on Microsoft's implementation of the Computer
Antivirus Research Organization (CARO) Naming Scheme and Malware Terminology. Based on
Microsoft Malware naming convntions, Microsoft Glossary, Microsoft Objective Criteria, and
CARO's definitions. Malware families are extracted from Microsoft SIRs since 2008 based on
Microsoft Malware, virus, and threat encyclopedia. Note that SIRs do NOT include all Microsoft
malware families.
23. MISP taxonomy to infer with MISP behavior or operation.
24. ms-caro-malware Malware Type and Platform classification based on Microsoft's implementation
of the Computer Antivirus Research Organization (CARO) Naming Scheme and Malware
Terminology.
25. NATO Classification Marking Marking of Classified and Unclassified materials as described by
the North Atlantic Treaty Organization, NATO.
26. Open Threat Taxonomy v1.1 (SANS) based on James Tarala of SANS
(http://www.auditscripts.com/resources/open_threat_taxonomy_v1.1a.pdf).
27. OSINT Open Source Intelligence - Classification
28. The Permissible Actions Protocol - or short: PAP PAP was designed to indicate how the received
information can be used. It's a protocol/taxonomy similar to TLP informing the recipients of
information what they can do with the received information.
29. Status of events used in Request Tracker.
30. Classification based on malware stealth techniques. Described in Introducing Stealth Malware
Taxonomy
31. TLP - Traffic Light Protocol The Traffic Light Protocol - or short: TLP - was designed with the
objective to create a favorable classification scheme for sharing sensitive information while
keeping the control over its distribution at the same time.
32. Vocabulary for Event Recording and Incident Sharing VERIS
A taxonomy contains a series of tags that can be used as normal tags in your MISP instance.
Tagging is a simple way to attach a classification to an event. In the early version of MISP, tagging
was local to an instance. Classification must be globally used to be efficient. After evaluating different
solutions of classification, we build a new scheme using the concept of machine tags.
Taxonomy is a classification of informations. Her, we classified Tags. Taxonomies are implemented in
a simple JSON format. Anyone can create their own taxonomy or reuse an existing one.
Taxonomys are in an independent git repository [https://github.com/MISP/misp-taxonomies]
These can be freely reused and integrated in other threat intel tools.
The advantage is that you even set a specific tag as being exportable. This means that you can
export your classification with other MISP instance and share the same taxonomies. Tagging is a
simple way to attach a classification to an event.
Classification must be globally used to be efficient.
If you want to enable a specific taxonomy, you can click on the cross to enable it.
Then you can even cherry-pick the tags you want to use on the system. If you want to use the whole
taxonomy, select all and then click on the cross in the top left.
Contributing to Taxonomy
It is quite easy. Create a JSON file describing your taxonomy as triple tags.
(e.g. check an existing one like Admiralty Scale), create a directory matching your name space, put
your machinetag file in the directory and pull your request. Publishing your taxonomy is as easy as a
simple git pull request on misp-taxonomies (https://github.com/MISP/misp-taxonomies). That's it.
Everyone can benefit from your taxonomy and can be automatically enabled in information sharing
tools like MISP.
Reserved Taxonomy
The following taxonomy namespaces are reserved and used internally to MISP.
galaxy mapping taxonomy with cluster:element:"value".
Adding Taxonomy in MISP
How are taxonomies integrated in MISP?
MISP administrator have only to import (or even cherry pick) the namespace or predicates they want
to use as tag.
Tags can be exported to other instances.
Tags are also accessible via the MISP REST API.
For more information, "Information Sharing and Taxonomies Practical Classification of Threat
Indicators using MISP" presentation given to the last MISP training in Luxembourg.
Adding a private taxonomy
$ cd /var/www/MISP/app/files/taxonomies/
$ mkdir privatetaxonomy
$ vi machinetag.json
Create a JSON file Create a JSON file describing your taxonomy as triple tags.
Once you are happy with your file go to MISP Web GUI taxonomies/index and update the taxonomies,
the newly created taxonomy should be visible, now you need to activate the tags within your
taxonomy.
How using Taxonomy in MISP
Filtering the distribution of events among MISP instances
Applying rules for distribution based on tags:
MISP Taxonomies - tools
machinetag.py is a parsing tool to dump taxonomies expressed in Machine Tags (Triple Tags) and list
all valid tags from a specific taxonomy.
% cd tools
% python machinetag.py
admiralty-scale:source-reliability="a"
admiralty-scale:source-reliability="b"
admiralty-scale:source-reliability="c"
admiralty-scale:source-reliability="d"
admiralty-scale:source-reliability="e"
admiralty-scale:source-reliability="f"
admiralty-scale:information-credibility="1"
admiralty-scale:information-credibility="2"
admiralty-scale:information-credibility="3"
admiralty-scale:information-credibility="4"
admiralty-scale:information-credibility="5"
admiralty-scale:information-credibility="6"
...
Other use cases using MISP taxonomies
Tags can be used to set events for further processing by external tools (e.g. VirusTotal autoexpansion using Viper).
Ensuring a classification manager classes the events before release (e.g. release of information from
air-gapped/classified networks).
Enriching IDS export with tags to fit your NIDS deployment.
MISP warning lists: The dilemma of false-positive
False-positive is a common issue in threat intelligence sharing.
It’s often a contextual issue:
false-positive might be different per community of users sharing information.
organization might have their own view on false-positive.
Based on the success of the MISP taxonomy model, we build misp-warninglists. They are lists of
well-known indicators that can be associated to potential false positives, errors or mistakes. They
are Simple JSON files.
The warning lists are integrated in MISP to display an info/warning box at the event and attribute level.
This can be enabled at MISP instance level. Default warning lists can be enabled or disabled like
known public resolver, multicast IP addresses, hashes for empty values, rfc1918, TLDs or known
google domains. The warning lists can be expanded or added in JSON locally or via pull requests
(https://github.com/MISP/misp-warninglists). Warning lists can be also used for critical or core
infrastructure warning, personally identifiable information...
Future functionalities related to MISP taxonomies
Sighting support (thanks to NCSC-NL) is integrated in MISP allowing to auto expire IOC based on
user detection.
Adjusting taxonomies (adding/removing tags) based on their score or visibility via sighting.
Simple taxonomy editors to help non-technical users to create their taxonomies.
Filtering mechanisms in MISP to rename or replace taxonomies/tags at pull and push
synchronisation.
More public taxonomies to be included
Galaxies
Managing Galaxies in MISP
Using Galaxies in MISP Events - Example
Available Galaxies
Clusters
Vocabularies
Galaxies
Galaxies in MISP are a method used to express a large object called cluster that can be attached to
MISP events or attributes. A cluster can be composed of one or more elements. Elements are
expressed as key-values.
There are default vocabularies available in MISP galaxy but those can be overwritten, replaced or
updated as you wish. Vocabularies are from existing standards (like STIX, Veris, MISP and so on) or
custom ones.
Existing clusters and vocabularies can be used as-is or as a template. MISP distribution can be
applied to each cluster to permit a limited or broader distribution scheme.
The objective is to have a common set of clusters for organizations starting analysis but that can be
expanded to localized information (which is not shared) or additional information (that can be shared).
MISP galaxy are available on Github.
Managing Galaxies in MISP
[warning] You need to have a specific role to manage Galaxies on a MISP instance.
Galaxies management is accessed using the Galaxies link on the top menu.
A list with all the galaxies existing on the server will appear.
Each galaxy can be explored using the icon at the end of the line.
Here is shown the metadata of the selected galaxy as well as a table with each available value as
well as some complementary data such as a description of the value or the activity, that is to say the
evolution of the use of each value.
Galaxies can be reimported from the submodules by cliking the "Update Galaxies" link on either the
galaxies list or while browsing a specific galaxy. A popup will appear to confirm the reimportation.
All galaxies will always be updated, even while browsing a specific galaxy.
Using Galaxies in MISP Events - Example
For this example, we will try to add a cluster to an existing event. This cluster will contains
informations about threat actor known as Sneaky Panda.
Here on the event view, we notice a blue frame under the metadatas with the title "Galaxies" and a
button "Add new cluster". Let's click on the latter to begin.
A popup will appear proposising to explore a particular galaxy or all at the same time. Here, as we
know we want to as a threat actor, we will choose the second option and scroll to find Sneaky Panda
(We are courageous, aren't we?).
Wait. No Sneaky Panda? Hm that's strange. Or maybe it is only registred as a alias. Let's have a look!
To do so we will use the search field which stay on top of the list. So what do we get? Beijing Group, is
it an alias of our threat actor.
Pointing the cursor on it will give us the answer.
We have a match. So we select it and here we go.
Clicking on the magnifying glass next to Threat actor redirects to the list of all threat actors Clicking on
the magnifying glass next to Beijing Group redirects us to a page about this group Clicking on the
addition symbole on the left of Beijing Group extends the module.
Available Galaxies
Clusters
Exploit-kit - Exploit-Kit is an enumeration of some exploitation kits used by adversaries. The list
includes document, browser and router exploit kits. It's not meant to be totally exhaustive but aim at
covering the most seen in the past 5 years.
Microsoft Activity Group - Activity groups as described by Microsoft.
Preventive Measure - Preventive measures.
Ransomware - Ransomware galaxy based on
https://docs.google.com/spreadsheets/d/1TWS238xacAtofLKh1n5uTsdijWdCEsGIM0Y0Hvmc5g/pubhtml
TDS - Traffic Direction System - TDS is a list of Traffic Direction System used by adversaries.
Threats Actors - Known or estimated adversary groups targeting organizations and employees.
Adversary groups are regularly confused with their initial operation or campaign. Threat actors are
characteristics of malicious actors (or adversaries) representing a cyber attack threat including
presumed intent and historically observed behaviour.
Tools - Enumeration of software tools used by adversaries. The list includes malware but also
common software regularly used by the adversaries.
Vocabularies
Common
certainty-level - Certainty level of an associated element or cluster
threat-actor
intended-effect - default STIX vocabulary for expressing the intended effect of a threat actor
motivation - default STIX vocabulary for expressing the motivation of a threat actor
planning-and-operational-support - default STIX vocabulary for expressing the planning and
operational support functions available to a threat actor.
sophistication - default STIX vocabulary for expressing the subjective level of sophistication of a threat
actor.
type - default STIX vocabulary for expressing the subjective type of a threat actor.
Sightings
Explanation
Using sightings on an event (GUI)
Advanced sightings
At Event level
Using sightings on an event (API)
Sightings
Basically, sighting is a system allowing people to react on attributes on an event. It was originally
designed to provide an easy method for user to tell when they see a given attribute, giving it more
credibility.
Now sightings have been improved to also provide a method to signal false positives, but also to give
an expiration date for some attributes.
Explanation
As said before, Sighting is a way for a user to say that they have seen or notice an attribute and
confirm its validity. An attribute can been spotted several times by the same user, that is why a single
user can use sighting several times on a single attribute.
Sometimes, some attributes can be considered as false positives, even if the false positive list do not
detect them (for instance, if the IDS flag is set to false) so they can also be notified. As well as
concerning sighting, the same user can signal a single attribute as a false positive several times.
It also happens that some attributes are only valid a certain time (for instance, in case of a phishing
campagne that is assumed to be up for only one week). In this case, people can also assign an
expiration date to an attribute, but this time, there can be only one valid expiration date per
organisation.
Using sightings on an event (GUI)
Sighting is applied to every attribute, under the column "Sightings", easily identifiable with its colored
number. This column shows three icons and three values.
These three values show respectively:
The number of sighting on the attribute, in green.
The number of times the attribute have been marked as false positive, in red.
The number of different expiration dates that have been affected on this attribute, in orange
Concerning the three icons:
The first one (Thumb up) allows to add a sighting on an attribute.
The second one (Thumb down) allows to mark the attribute as a false positive.
The third one (Tool) opens a popup for advanced sightings, showing sightings details and
allowing different actions.
Advanced sightings
The first tab, "Graph", represents a line graph showing the evolution of sightings and false
positives over time.
The second tab gives a quick view of all the sightings applied to the attribute.
The third tab gives a quick view of the sightings applied to the attribute by your own organisation
only.
The last tab can be used to add either a sighting, mark the attribute as a false positive, or define
an expiration date. You can precise both the date and time of day, as well as note a particular
source where the sighting comes from.
At Event level
The total number of sightings is also visible as part of the metadata in front of the Sightings label, as
well as a sparkline graph that summarize the evolution of sightings.
Clicking on the tool will show sighting details for the whole event.
Using sightings on an event (API)
WiP
MISP Attribute Categories vs Types
Categories
Types
MISP Attribute Categories vs Types
Payload
delivery
Artifacts
dropped
md5
X
X
sha1
X
X
sha256
X
X
filename
X
X
Category
Internal
reference
Targeting
data
Antivirus
detection
pdb
X
filename|md5
X
X
filename|sha1
X
X
filename|sha256
X
X
ip-src
X
ip-dst
X
hostname
X
domain
X
domain|ip
email-src
X
email-dst
X
email-subject
X
email-attachment
X
url
X
http-method
user-agent
X
regkey
X
regkey|value
X
AS
X
snort
pattern-in-file
X
X
pattern-in-traffic
X
pattern-in-memory
X
yara
X
vulnerability
X
attachment
X
malware-sample
link
X
comment
X
text
other
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
named pipe
X
mutex
X
target-user
X
target-email
X
target-machine
X
target-org
X
target-location
X
target-external
X
btc
iban
bic
bank-account-nr
aba-rtn
bin
cc-number
prtn
threat-actor
campaign-name
campaign-id
malware-type
X
uri
authentihash
X
X
ssdeep
X
X
imphash
X
X
pehash
X
sha224
X
X
sha384
X
X
sha512
X
X
sha512/224
X
X
sha512/256
X
X
tlsh
X
filename|authentihash
X
X
filename|ssdeep
X
X
filename|imphash
X
X
filename|pehash
X
X
filename|sha224
X
X
filename|sha384
X
X
filename|sha512
X
X
filename|sha512/224
X
X
filename|sha512/256
X
X
filename|tlsh
X
X
windows-scheduledtask
X
windows-servicename
X
windows-servicedisplayname
X
whois-registrant-email
whois-registrantphone
whois-registrantname
whois-registrar
whois-creation-date
x509-fingerprint-sha1
dns-soa-email
X
X
size-in-bytes
counter
datetime
cpe
port
ip-dst|port
X
ip-src|port
X
hostname|port
X
email-dst-displayname
X
email-src-displayname
X
email-header
X
email-reply-to
X
email-x-mailer
X
email-mime-boundary
X
email-thread-index
X
email-message-id
X
github-username
github-repository
github-organisation
jabber-id
twitter-id
first-name
middle-name
last-name
date-of-birth
place-of-birth
gender
passport-number
passport-country
passport-expiration
redress-number
nationality
visa-number
issue-date-of-the-visa
primary-residence
country-of-residence
special-servicerequest
frequent-flyer-number
travel-details
payment-details
place-port-of-originalembarkation
place-port-ofclearance
place-port-of-onwardforeign-destination
passenger-namerecord-locatornumber
mobile-application-id
Category
X
Persistence
mechanism
Network
activity
Payload
type
Attribution
External
analysis
md5
X
sha1
X
sha256
X
filename
X
X
pdb
filename|md5
X
filename|sha1
X
filename|sha256
X
ip-src
X
X
ip-dst
X
X
hostname
X
X
domain
X
X
domain|ip
X
X
email-src
email-dst
X
email-subject
email-attachment
url
X
http-method
X
user-agent
X
X
X
regkey
X
X
regkey|value
X
X
AS
X
X
snort
X
X
pattern-in-file
X
X
pattern-in-traffic
X
X
pattern-in-memory
X
yara
vulnerability
X
attachment
X
X
malware-sample
X
link
X
comment
X
X
X
X
X
text
X
X
X
X
X
other
X
X
X
X
X
named pipe
mutex
target-user
target-email
target-machine
target-org
target-location
target-external
btc
iban
bic
bank-account-nr
aba-rtn
bin
cc-number
prtn
threat-actor
X
campaign-name
X
campaign-id
X
malware-type
uri
authentihash
ssdeep
imphash
pehash
sha224
sha384
sha512
sha512/224
sha512/256
tlsh
filename|authentihash
filename|ssdeep
filename|imphash
filename|pehash
filename|sha224
filename|sha384
filename|sha512
filename|sha512/224
filename|sha512/256
X
filename|tlsh
windows-scheduledtask
windows-servicename
windows-servicedisplayname
whois-registrant-email
X
whois-registrantphone
X
whois-registrantname
X
whois-registrar
X
whois-creation-date
X
x509-fingerprint-sha1
X
X
X
dns-soa-email
size-in-bytes
counter
datetime
cpe
port
ip-dst|port
X
X
ip-src|port
X
X
hostname|port
email-dst-displayname
email-src-displayname
email-header
email-reply-to
email-x-mailer
email-mime-boundary
email-thread-index
email-message-id
github-username
github-repository
github-organisation
jabber-id
twitter-id
first-name
middle-name
last-name
date-of-birth
place-of-birth
gender
passport-number
passport-country
passport-expiration
redress-number
nationality
visa-number
issue-date-of-the-visa
primary-residence
country-of-residence
special-servicerequest
frequent-flyer-number
travel-details
payment-details
place-port-of-originalembarkation
place-port-ofclearance
place-port-of-onwardforeign-destination
passenger-namerecord-locatornumber
X
mobile-application-id
Category
Support
Tool
Social
network
md5
sha1
sha256
filename
pdb
filename|md5
filename|sha1
filename|sha256
ip-src
ip-dst
hostname
domain
domain|ip
email-src
X
email-dst
X
email-subject
email-attachment
url
http-method
user-agent
regkey
regkey|value
AS
snort
pattern-in-file
pattern-in-traffic
pattern-in-memory
yara
vulnerability
Person
Other
attachment
X
malware-sample
link
X
comment
X
X
X
X
text
X
X
X
X
other
X
X
X
X
named pipe
mutex
target-user
target-email
target-machine
target-org
target-location
target-external
btc
iban
bic
bank-account-nr
aba-rtn
bin
cc-number
prtn
threat-actor
campaign-name
campaign-id
malware-type
uri
authentihash
ssdeep
imphash
pehash
sha224
sha384
sha512
sha512/224
sha512/256
tlsh
filename|authentihash
filename|ssdeep
filename|imphash
filename|pehash
filename|sha224
filename|sha384
filename|sha512
filename|sha512/224
filename|sha512/256
filename|tlsh
windows-scheduled-task
windows-service-name
windows-service-displayname
whois-registrant-email
whois-registrant-phone
whois-registrant-name
whois-registrar
whois-creation-date
x509-fingerprint-sha1
dns-soa-email
size-in-bytes
X
counter
X
datetime
X
cpe
X
port
X
ip-dst|port
ip-src|port
hostname|port
email-dst-display-name
email-src-display-name
email-header
email-reply-to
email-x-mailer
email-mime-boundary
email-thread-index
email-message-id
github-username
X
github-repository
X
github-organisation
X
jabber-id
X
twitter-id
X
first-name
X
middle-name
X
last-name
X
date-of-birth
X
place-of-birth
X
gender
X
passport-number
X
passport-country
X
passport-expiration
X
redress-number
X
nationality
X
visa-number
X
issue-date-of-the-visa
X
primary-residence
X
country-of-residence
X
special-service-request
X
frequent-flyer-number
X
travel-details
X
payment-details
X
place-port-of-original-embarkation
X
place-port-of-clearance
X
place-port-of-onward-foreigndestination
X
passenger-name-record-locatornumber
X
mobile-application-id
Categories
Internal reference: Reference used by the publishing party (e.g. ticket number)
Targeting data: Targeting information to include recipient email, infected machines, department,
and or locations.
Antivirus detection: List of anti-virus vendors detecting the malware or information on detection
performance (e.g. 13/43 or 67%). Attachment with list of detection or link to VirusTotal could be
placed here as well.
Payload delivery: Information about the way the malware payload is initially delivered, for
example information about the email or web-site, vulnerability used, originating IP etc. Malware
sample itself should be attached here.
Artifacts dropped: Any artifact (files, registry keys etc.) dropped by the malware or other
modifications to the system
Payload installation: Location where the payload was placed in the system and the way it was
installed. For example, a filename|md5 type attribute can be added here like this:
c:\windows\system32\malicious.exe|41d8cd98f00b204e9800998ecf8427e.
Persistence mechanism: Mechanisms used by the malware to start at boot. This could be a
registry key, legitimate driver modification, LNK file in startup
Network activity: Information about network traffic generated by the malware
Payload type: Information about the final payload(s). Can contain a function of the payload, e.g.
keylogger, RAT, or a name if identified, such as Poison Ivy.
Attribution: Identification of the group, organisation, or country behind the attack
External analysis: Any other result from additional analysis of the malware like tools output
Examples: pdf-parser output, automated sandbox analysis, reverse engineering report.
Financial fraud: Financial Fraud indicators, for example: IBAN Numbers, BIC codes, Credit card
numbers, etc.
Support Tool: Tools supporting analysis or detection of the event
Social network: Social networks and platforms
Person: A human being - natural person
Other: Attributes that are not part of any other category or are meant to be used as a component
in MISP objects in the future
Types
md5: You are encouraged to use filename|md5 instead. A checksum in md5 format, only use this
if you don't know the correct filename
sha1: You are encouraged to use filename|sha1 instead. A checksum in sha1 format, only use
this if you don't know the correct filename
sha256: You are encouraged to use filename|sha256 instead. A checksum in sha256 format,
only use this if you don't know the correct filename
filename: Filename
pdb: Microsoft Program database (PDB) path information
filename|md5: A filename and an md5 hash separated by a | (no spaces)
filename|sha1: A filename and an sha1 hash separated by a | (no spaces)
filename|sha256: A filename and an sha256 hash separated by a | (no spaces)
ip-src: A source IP address of the attacker
ip-dst: A destination IP address of the attacker or C&C server. Also set the IDS flag on when this
IP is hardcoded in malware
hostname: A full host/dnsname of an attacker. Also set the IDS flag on when this hostname is
hardcoded in malware
domain: A domain name used in the malware. Use this instead of hostname when the upper
domain is important or can be used to create links between events.
domain|ip: A domain name and its IP address (as found in DNS lookup) separated by a | (no
spaces)
email-src: The email address (or domainname) used to send the malware.
email-dst: A recipient email address that is not related to your constituency.
email-subject: The subject of the email
email-attachment: File name of the email attachment.
url: url
http-method: HTTP method used by the malware (e.g. POST, GET, ...).
user-agent: The user-agent used by the malware in the HTTP request.
regkey: Registry key or value
regkey|value: Registry value + data separated by |
AS: Autonomous system
snort: An IDS rule in Snort rule-format. This rule will be automatically rewritten in the NIDS
exports.
pattern-in-file: Pattern in file that identifies the malware
pattern-in-traffic: Pattern in network traffic that identifies the malware
pattern-in-memory: Pattern in memory dump that identifies the malware
yara: Yara signature
vulnerability: A reference to the vulnerability used in the exploit
attachment: Please upload files using the Upload Attachment button.
malware-sample: Please upload files using the Upload Attachment button.
link: Link to an external information
comment: Comment or description in a human language. This will not be correlated with other
attributes
text: Name, ID or a reference
other: Other attribute
named pipe: Named pipe, use the format .\pipe\
mutex: Mutex, use the format \BaseNamedObjects\
target-user: Attack Targets Username(s)
target-email: Attack Targets Email(s)
target-machine: Attack Targets Machine Name(s)
target-org: Attack Targets Department or Organization(s)
target-location: Attack Targets Physical Location(s)
target-external: External Target Organizations Affected by this Attack
btc: Bitcoin Address
iban: International Bank Account Number
bic: Bank Identifier Code Number
bank-account-nr: Bank account number without any routing number
aba-rtn: ABA routing transit number
bin: Bank Identification Number
cc-number: Credit-Card Number
prtn: Premium-Rate Telephone Number
threat-actor: A string identifying the threat actor
campaign-name: Associated campaign name
campaign-id: Associated campaign ID
malware-type:
uri: Uniform Resource Identifier
authentihash: You are encouraged to use filename|authentihash instead. Authenticode
executable signature hash, only use this if you don't know the correct filename
ssdeep: You are encouraged to use filename|ssdeep instead. A checksum in the SSDeep format,
only use this if you don't know the correct filename
imphash: You are encouraged to use filename|imphash instead. A hash created based on the
imports in the sample, only use this if you don't know the correct filename
pehash: PEhash - a hash calculated based of certain pieces of a PE executable file
sha224: You are encouraged to use filename|sha224 instead. A checksum in sha224 format,
only use this if you don't know the correct filename
sha384: You are encouraged to use filename|sha384 instead. A checksum in sha384 format,
only use this if you don't know the correct filename
sha512: You are encouraged to use filename|sha512 instead. A checksum in sha512 format,
only use this if you don't know the correct filename
sha512/224: You are encouraged to use filename|sha512/224 instead. A checksum in
sha512/224 format, only use this if you don't know the correct filename
sha512/256: You are encouraged to use filename|sha512/256 instead. A checksum in
sha512/256 format, only use this if you don't know the correct filename
tlsh: You are encouraged to use filename|tlsh instead. A checksum in the Trend Micro Locality
Sensitive Hash format, only use this if you don't know the correct filename
filename|authentihash: A checksum in md5 format
filename|ssdeep: A checksum in ssdeep format
filename|imphash: Import hash - a hash created based on the imports in the sample.
filename|pehash: A filename and a PEhash separated by a |
filename|sha224: A filename and a sha-224 hash separated by a |
filename|sha384: A filename and a sha-384 hash separated by a |
filename|sha512: A filename and a sha-512 hash separated by a |
filename|sha512/224: A filename and a sha-512/224 hash separated by a |
filename|sha512/256: A filename and a sha-512/256 hash separated by a |
filename|tlsh: A filename and a Trend Micro Locality Sensitive Hash separated by a |
windows-scheduled-task: A scheduled task in windows
windows-service-name: A windows service name. This is the name used internally by windows.
Not to be confused with the windows-service-displayname.
windows-service-displayname: A windows service's displayname, not to be confused with the
windows-service-name. This is the name that applications will generally display as the service's
name in applications.
whois-registrant-email: The e-mail of a domain's registrant, obtained from the WHOIS
information.
whois-registrant-phone: The phone number of a domain's registrant, obtained from the WHOIS
information.
whois-registrant-name: The name of a domain's registrant, obtained from the WHOIS
information.
whois-registrar: The registrar of the domain, obtained from the WHOIS information.
whois-creation-date: The date of domain's creation, obtained from the WHOIS information.
x509-fingerprint-sha1: X509 fingerprint in SHA-1 format
dns-soa-email: RFC1035 mandates that DNS zones should have a SOA (Statement Of Authority)
record that contains an email address where a PoC for the domain could be contacted. This can
sometimes be used for attribution/linkage between different domains even if protected by whois
privacy
size-in-bytes: Size expressed in bytes
counter: An integer counter, generally to be used in objects
datetime: Datetime in the ISO 8601 format
cpe: Common platform enumeration
port: Port number
ip-dst|port: IP destination and port number seperated by a |
ip-src|port: IP source and port number seperated by a |
hostname|port: Hostname and port number seperated by a |
email-dst-display-name: Email destination display name
email-src-display-name: Email source display name
email-header: Email header
email-reply-to: Email reply to header
email-x-mailer: Email x-mailer header
email-mime-boundary: The email mime boundary separating parts in a multipart email
email-thread-index: The email thread index header
email-message-id:
github-username: A github user name
github-repository: A github repository
github-organisation: A github organisation
jabber-id: Jabber ID
twitter-id: Twitter ID
first-name: First name of a natural person
middle-name: Middle name of a natural person
last-name: Last name of a natural person
date-of-birth: Date of birth of a natural person (in YYYY-MM-DD format)
place-of-birth: Place of birth of a natural person
gender: The gender of a natural person (Male, Female, Other, Prefer not to say)
passport-number: The passport number of a natural person
passport-country: The country in which the passport was issued
passport-expiration: The expiration date of a passport
redress-number: The Redress Control Number is the record identifier for people who apply for
redress through the DHS Travel Redress Inquiry Program (DHS TRIP). DHS TRIP is for travelers
who have been repeatedly identified for additional screening and who want to file an inquiry to
have erroneous information corrected in DHS systems
nationality: The nationality of a natural person
visa-number: Visa number
issue-date-of-the-visa: The date on which the visa was issued
primary-residence: The primary residence of a natural person
country-of-residence: The country of residence of a natural person
special-service-request: A Special Service Request is a function to an airline to provide a
particular facility for A Passenger or passengers.
frequent-flyer-number: The frequent flyer number of a passenger
travel-details: Travel details
payment-details: Payment details
place-port-of-original-embarkation: The orignal port of embarkation
place-port-of-clearance: The port of clearance
place-port-of-onward-foreign-destination: A Port where the passenger is transiting to
passenger-name-record-locator-number: The Passenger Name Record Locator is a key under
which the reservation for a trip is stored in the system. The PNR contains, among other data, the
name, flight segments and address of the passenger. It is defined by a combination of five or six
letters and numbers.
mobile-application-id: The application id of a mobile application
Sharing / Synchronisation
Concept
Setup
Adding a server
Test connection
Rules
Troubleshooting
Collaboration
Proposals
Forums / Threats
Comments to events
Contact a reporter
Receive alerts
Events
Sharing-groups
Recommendation
MISP Staging System
MISP SECOps System
Sharing / Synchronisation
Explanation
Setup
Roles
Tools
Server Settings
Events
Sharing groups
Recommendations
MISP's core functionality is sharing where everyone can be a consumer and/or a
contributor/producer.
Quick benefit without the obligation to contribute
Low barrier access to get acquainted to the system
Concept
The following figure shows the concept how different MISP instances could tie together.
Setup
Adding a server
Servers can be added by users via
https://<misp url>/servers/add
The Add Server Form has several input fields:
1. Base URL
The base-url to the external server you want to sync with. Example: https://foo.sig.mil.be
2. Instance Name
A name that will make it clear to your users what this instance is. For example: Organisation A's
instance
3. Remote Sync Organisation Type
The organization having the external server you want to sync with. Example: BE
4. Local Organisation
This setting will configure which organisation will be assigned to the events being pulled.
5. Authkey
You can find the authentication key on your profile on the external server.
6. Push
Allow the upload of events and their attributes. That means only Events that match the given filter
will be pushed to the server.
E.g. it can limit push of events to events not being TLP:RED
1. Pull
Allow the download of events and their attributes from the server. That means only Events
matching the given criteria will be pulled.
E.g. it can limit to NOT download Type:OSINT events.
2. Self Signed
Click this, if you would like to allow a connection despite the other instance using a self-signed
certificate (not recommended). (server certificate file still needed)
3. Server certificate file
You can also upload a certificate file if the instance you are trying to connect to has its own
signing authority. (*.pem)
4. Client certificate file
You can also upload a certificate file if the instance you are trying to connect to has its own
signing authority. (*.pem)
Test connection
Test connection can be used to test the connection to the remote server and will give a feedback
about local and remote version of MISP.
Rules
Rules are used to limit sharing to e.g. events with a given tag, or disabling sharing for events
containing a certain Tag.
Troubleshooting
If you have issues connecting to a remote servers try to do the following things:
try to connect with your user account to the remote server, to ensure the password is still valid
and that your API key is valid
try to connect with your user account to the remote server and check your roles on the remote
server
with connection issues do a package capture to find out more
if you have a SSL connection issue to a remote server with a signed by a CA that is not included
in OS, make sure the whole certificate path is included in the path.
Collaboration
Proposals
Proposals can be used to propose new attribute values that can be reviewed by the event owner.
Forums / Threats
Forums can be used to discuss non event related topics.
Discussions can be accessed on the top "Global Actions - List Discussions"
Discussions will and can not be shared with other servers
and via URL:
https://<misp url>/threads/index
Create a new Topic
To create a new topic
https://<misp url>/posts/add
Comment a topic
A topic can be commented by any user
https://<misp url>/threads/view/<topic id>
Comments to events
In MISP ongoing events can be commented by every user to ask free text question to events.
Comments to events will not be shared with other servers
Contact a reporter
This feature can be used to contact the person or the organisation that the person belongs to that has
created the event.
All E-Mails can be enforced to be encrypted
Receive alerts
It is possible to get alerts via encrypted mail in the following cases:
published events by other user of the MISP instance
events pushed to the MISP instance
events pulled by the MISP instance
These E-Mail alerts are an opt-in feature
Events
This will describe what to do within events to be shared.
Only events that are published will be shared
Sharing-groups
There is an article about sharing groups in here
Recommendation
The following section will describe what is the best practice how many MISP instances that showed to
be good for orgs. Of course depending on your specific requirements an architecture could be more
spread or simplified.
The architecture is divided into several systems / stages beginning with:
MISP Staging System
This systems purpose is to be linked to all available external MISP systems that you have access to. It
will download all events and do enrichment between these events.
MISP SECOps System
This system is the main system used by human analysts. It will it is not linked to any external MISP
instance other then the Staging System.
To publish events to the community assign the right tags to match your push Rules and publish the
event
MISP ZeroMQ
MISP ZeroMQ configuration
MISP ZeroMQ debugging and testing
Testing with sub.py tool
MISP ZeroMQ
MISP includes a flexible publish-subscribe model to allow real-time integration of the MISP activities
(event publication, attribute creation or removal, sighting). The MISP ZeroMQ plugin operates at
global level in MISP which means standard distribution rules don't apply and every activities will be
published within the ZeroMQ pub-sub channels.
MISP ZeroMQ functionality can be used for various model of integration or to extend MISP
functionalities:
real-time search of indicators into a SIEM
automatic expansion
dashboard activities
logging mechanisms
continuous indexing
custom software or scripting
The following notification topic channels exist and can be included in the MISP ZeroMQ pub-sub:
misp_json
- events published
misp_json_attribute
misp_json_sighting
misp_json_user
- attribute updated or created
- sighting added to an attribute or an event
- user updates or creation
misp_json_organisation
misp_json_self
- organisation updates or creation
- keep-alive messages sent every minute
MISP ZeroMQ configuration
To enable MISP ZeroMQ, the feature must be enabled in the Plugin setting tab.
Each notification channels can be enabled (from event publication to sightings), the MISP site admin
can decide which type of message to publish.
By default, the ZMQ pub-sub channel is available to localhost only on TCP port 50000. The binding of
the pub-sub channel can be updated in the configuration interface as shown above
MISP ZeroMQ debugging and testing
In the diagnostic section, ZeroMQ service can be started and stopped. There is a small status option
to give information about the numbers of events processed by the service.
Testing with sub.py tool
A simple command line tool is included with MISP to connect to the MISP ZeroMQ channel and get
the notifications:
python3 sub.py --help
usage: sub.py [-h] [-s] [-p PORT] [-r HOST] [-o ONLY] [-t SLEEP]
Generic ZMQ client to gather events, attributes and sighting updates from a
MISP instance
optional arguments:
-h, --help
show this help message and exit
-s, --stats
print regular statistics on stderr
-p PORT, --port PORT
set TCP port of the MISP ZMQ (default: 50000)
-r HOST, --host HOST
set host of the MISP ZMQ (default: 127.0.0.1)
-o ONLY, --only ONLY
set filter (misp_json, misp_json_attribute or
misp_json_sighting) to limit the output a specific
type (default: no filter)
-t SLEEP, --sleep SLEEP
sleep time (default: 2)
The
sub.py
will output the JSON objects for the subscribed topic, by default, all the topic channels
are dumped:
[email protected]:/var/www/MISP/tools/misp-zmq$ python3 -u sub.py
....
{
"uptime": 50,
"status": "And when you're dead I will be still alive."
}
{
"uptime": 60,
"status": "And believe me I am still alive."
}
{
"uptime": 70,
"status": "I'm doing science and I'm still alive."
}
{
"uptime": 80,
"status": "I feel FANTASTIC and I'm still alive."
}
{
"uptime": 90,
"status": "While you're dying I'll be still alive."
}
{
"Sighting": {
"uuid": "592d9588-fda0-490f-bf6e-4e56950d210f",
"source": "",
"type": "0",
"date_sighting": 1496159624,
"org_id": "2",
"event_id": "8102",
"attribute_id": "1044812"
}
}
{
"Attribute": {
"id": "1044802",
"value2": "",
"value1": "1.2.3.4",
"uuid": "592d8494-7120-4760-b5e2-4858950d210f",
"batch_import": "0",
"comment": "",
"value": "1.2.3.4",
"type": "ip-dst",
"to_ids": 0,
"timestamp": 1496155284,
"distribution": "5",
"sharing_group_id": 0,
"deleted": "0",
"disable_correlation": "0",
"event_id": "8100",
"category": "Network activity"
}
}
....
| jq .
Appendix A: External Authentication
The external authentication mechanism described
The external authentication allows a user or an external tool to authenticate with MISP using an
arbitrary value passed along in a custom header. This authentication method overrides the regular
authentication mechanisms and is customisable by a site-admin.
It is possible to create a mixed mode MISP setup where certain users can go through the normal
authentication mechanism and other users are required to use the external authentication method.
Setting up the external authentication mechanism
To change the authentication settings, navigate to Administration - Server settings - Plugin settings
The settings associated with the external authentication can be found by pressing the CustomAuth
button as depicted below:
To change a setting simply double click on the value to edit the field. Use the guidance provided by
the setting tool to configure the external authentication. The accessible settings are as follows:
enable: Enable or disable external authentication (off by default)
header: The header which MISP will use to identify users
required: Enabling this setting will force all users to use the external authentication. Leave this
disabled allows administrators to assign external authentication or regular authentication users.
only_allow_source: Setting a url / IP address here will only allow requests that originated from
the given address
name: The name to be used for the authentication mechanism. This is reflected in the user
creation / edit views, the logs and the error messages on failed logins.
disable_logout: Disable the default logout button. Using an external authentication mechanism
that authenticates via the header with each requests makes the logout button obsolete.
custom_password_reset: If your authentication system has a url that a user can access to reset
his/her password, please specify the full url for it here. This will then be reused in the UI.
custom_password_logout: If your authentication system has a url that a user can access to
logout, please specify the full url for it here. This will then be reused in the UI.
User management
Using a new setting, user self management can be disabled for all users that are not administrators
via the MISP.disableUserSelfManagement setting, found in the MISP settings tab. Enabling this
setting removes the ability of users to change their user settings and reset their authentication keys.
All other functionality remains unchanged.
To create an external authenticated user, simply tick the External authentication user checkbox, after
which an external auth key field will appear. This will be used to identify the users via the passed
along header.
Logging
For a description of the logging facilities provided by this plugin, please refer to the "Logging of failed
authentication attempts" section of the Administration section.
Appendix B: ACL descriptors
Querying the ACL system
MISP allows site admins to query the ACL system for various types of data. This can be interesting
when tuning for example WAF access to MISP. All applicable queries can be requested via
/servers/queryACL
Getting a list of URLs accessible to a role
https://<misp url>/servers/queryACL/printRoleAccess/<role id>
The above URL will return a JSON with all accessible URLs for the given role ID. If no Role ID is
provided, a JSON containing all roles and their access lists will be returned.
Example:
{
"2": {
"name": "User",
"urls": [
"/attributes/add/*",
"/attributes/add_attachment/*",
"/attributes/add_threatconnect/*",
"/attributes/attributeReplace/*",
"/attributes/delete/*",
"/attributes/deleteSelected/*",
"/attributes/download/*",
"/attributes/downloadAttachment/*",
"/attributes/downloadSample/*",
"/attributes/edit/*",
"/attributes/editField/*",
"/attributes/editSelected/*",
"/attributes/fetchEditForm/*",
"/attributes/fetchViewValue/*",
"/attributes/hoverEnrichment/*",
"/attributes/index/*",
"/attributes/restSearch/*",
"/attributes/returnAttributes/*",
"/attributes/rpz/*",
"/attributes/search/*",
"/attributes/searchAlternate/*",
"/attributes/text/*",
"/attributes/updateAttributeValues/*",
"/attributes/view/*",
"/eventDelegations/acceptDelegation/*",
"/eventDelegations/delegateEvent/*",
"/eventDelegations/deleteDelegation/*",
"/eventDelegations/view/*",
"/events/add/*",
"/events/addIOC/*",
"/events/addTag/*",
"/events/add_misp_export/*",
"/events/contact/*",
"/events/csv/*",
"/events/delegation_index/*",
"/events/delete/*",
"/events/downloadExport/*",
"/events/downloadOpenIOCEvent/*",
"/events/downloadSearchResult/*",
"/events/edit/*",
"/events/export/*",
"/events/exportChoice/*",
"/events/filterEventIndex/*",
"/events/freeTextImport/*",
"/events/hids/*",
"/events/index/*",
"/events/nids/*",
"/events/proposalEventIndex/*",
"/events/queryEnrichment/*",
"/events/removePivot/*",
"/events/removeTag/*",
"/events/restSearch/*",
"/events/saveFreeText/*",
"/events/stix/*",
"/events/updateGraph/*",
"/events/view/*",
"/events/viewEventAttributes/*",
"/events/viewGraph/*",
"/events/xml/*",
"/jobs/cache/*",
"/jobs/getGenerateCorrelationProgress/*",
"/jobs/getProgress/*",
"/logs/event_index/*",
"/logs/maxDateActivity/*",
"/logs/returnDates/*",
"/organisations/fetchOrgsForSG/*",
"/organisations/fetchSGOrgRow/*",
"/organisations/index/*",
"/organisations/landingpage/*",
"/organisations/view/*",
"/pages/display/*",
"/posts/add/*",
"/posts/delete/*",
"/posts/edit/*",
"/regexp/index/*",
"/roles/index/*",
"/roles/view/*",
"/servers/fetchServersForSG/*",
"/shadowAttributes/accept/*",
"/shadowAttributes/acceptSelected/*",
"/shadowAttributes/add/*",
"/shadowAttributes/add_attachment/*",
"/shadowAttributes/delete/*",
"/shadowAttributes/discard/*",
"/shadowAttributes/discardSelected/*",
"/shadowAttributes/download/*",
"/shadowAttributes/edit/*",
"/shadowAttributes/editField/*",
"/shadowAttributes/fetchEditForm/*",
"/shadowAttributes/index/*",
"/shadowAttributes/view/*",
"/sharingGroups/index/*",
"/sharingGroups/view/*",
"/sightings/add/*",
"/sightings/delete/*",
"/tags/add/*",
"/tags/delete/*",
"/tags/edit/*",
"/tags/index/*",
"/tags/quickAdd/*",
"/tags/selectTag/*",
"/tags/selectTaxonomy/*",
"/tags/showEventTag/*",
"/tags/view/*",
"/tags/viewTag/*",
"/taxonomies/index/*",
"/taxonomies/taxonomyMassConfirmation/*",
"/taxonomies/view/*",
"/templateElements/index/*",
"/templates/deleteTemporaryFile/*",
"/templates/index/*",
"/templates/populateEventFromTemplate/*",
"/templates/submitEventPopulation/*",
"/templates/templateChoices/*",
"/templates/uploadFile/*",
"/templates/view/*",
"/threads/index/*",
"/threads/view/*",
"/threads/viewEvent/*",
"/users/dashBoard/*",
"/users/downloadTerms/*",
"/users/edit/*",
"/users/histogram/*",
"/users/index/*",
"/users/login/*",
"/users/logout/*",
"/users/memberslist/*",
"/users/resetauthkey/*",
"/users/routeafterlogin/*",
"/users/statistics/*",
"/users/terms/*",
"/users/updateLoginTime/*",
"/users/view/*",
"/whitelists/index/*"
]
}
}
Getting a list of all accessible controllers and actions in MISP
https://<misp url>/servers/queryACL/printAllFunctionNames
This URL will return a JSON with all controller and all mapped functions within them.
Viewing a list of yet unmapped functions
https://<misp url>/servers/queryACL/findMissingFunctionNames
Functions that have not been tied into the new ACL yet show up here. These functions will (until
added to the ACL) only be accessible to site admins.
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement