Articles

    These are search examples that can be copied and modified to fit most common search scenarios.

    Search Syntax - A valid search can be a list of search terms to match in the data. Analytics are included by adding a pipe symbol.

    SEARCH TERMS | FUNCTIONS

    Simple Searches

    ERROR OR WARN OR FATAL |

    This will search all files, across all hosts for ERRORS,WARNINGS and FATAL messages.

    ERROR OR WARN OR FATAL | _host.count(_filename)

    Searches for all ERROR,WARN and FATALs in the data and then groups the results by the originating file and host

    | _tag.equals(java) _tag.equals(jboss)

    This will search all data tagged with java AND jboss

    | _tag.contains(java,jboss)

    This will search all data tagged with java OR jboss

    | include(WARN,FATAL,ERROR) _host.include(svr-www01,svr-www1) _filename.equals(access.log)

    The includes and excludes functions have an implicit AND between them. The parameters to these functions are a list of strings to include or exclude these have an implicit OR between. This search translates to: include all events that contain WARN OR FATAL OR ERROR on hosts SVR-WWW01 or SVR-WWW02 AND in the file access.log

    | _tag.contains(java,jboss)

    This will search all data tagged with java OR jboss

    Exception

    This will do a basic keyword search for Exceptions in your environment.

    | include(ERROR,WARN,FATAL)

    This will search all files, across all hosts for ERRORS,WARNINGS and FATAL messages.

    | exclude(DEBUG,INFO)

    This will search all files, across all hosts excluding DEBUG and INFO messages.

    * | _level.count(_host)

    This will search all files, across hosts and count the different log levels

    Group By Fields

    | _tag.equals(weblogs) clientIp.count(site,)

    This search will give you the number different client ip addresses broken down by the site accessed.site is the group by field in this example.

    | _tag.equals(weblogs) downloads.count(site+service,)

    The search '+' notation will concatenate two fields together. This can be used to write more advanced group-by searches without extending your datatype. In this example the number of downloads is broken down by the website AND the service accessed. Since our data source aggregates service statistics from all our websites getting the counts broken down by service may not be enough.

    | _tag.equals(weblogs) pagetype.equals(zip) site+service.count()

    The '+' notation can also be used to create a new unique field. THis example counts the number zip files for each service on each site being monitored.

    Find by TAG

    * | _tag.equals(syslog)

    Returns all syslog data

    WARN | _type.equals(log4j) _tag.include(XXX)

    Search java application log data and .NET applications that use log4net for WARNINGS. Only files from datasources containing the string XXX is considered

    | _type.equals(log4j) not(INFO)

    Filters out all the INFO messages from the log4j datatype

    Find Data by Filename or Path

    * | _filename.include(error.log)

    Searches for the error.log. This search will also include rolling files

    * | _path.include(/var/log)

    Returns all the files being watched in the Linux system log folder. If you have apache logs and mysql logs, this search will also return folders located in /var/log.

    Find by Host or Agent

    ERROR OR WARN OR FATAL | _host.count()

    This Search will return all the ERROR,WARNING and FATAL messages group by the host.

    ERROR | _host.equals(www-cluster)

    Returns all ERROR events from only the www-cluster host.

    404 | _host.include(www)

    Returns all 404 occuring on hosts containing 'www' in the hostname. Using _host.contains(XXX) is useful when you have multiple hosts that form a logical group where the name reflects the role or location of the server.

    REBOOT | _agent.contains(Forwarder)

    Returns all the REBOOT messages found on the Forwarders. The _agent field allows you to search against the role of the agent.i.e Management, Forwarders, Indexers etc.

    404 | _host.contains(webfarm-) _host.not(dev,uat)

    This search will return all the 404 events from all hosts that contain webfarm-* but not from webfarm-dev or webfarm-uat

    | _agent.include(lab.uk) _host.count()

    Returns all the events coming from hosts belonging to the lab.uk.* zone and then breaks down the results by host.

    Search by Type

    | _type.equals(log4j)

    The volume of events coming from log4j files.

    | _type.equals(log4j) level.equals(ERROR)

    All ERROR messages but only from the log4j files.

    | _type.equals(log4j) _filename.count()

    This returns all the log4j events broken down by the filename.

    | _type.include(www) request.contains(signup.html) UserAgent.count()

    Returns all the user agents that made the request to the signup page

    Exception | _type.equals(log4j)

    This a keyword search that finds the 'exception' substring in your log4j files
    Keyword Searches using patterns

    (*)Exception | 1.count()

    (*) expression will match the block of text before the substring 'Exception'. In most cases this will be an exception.

    ([A-Za-z\.]+)Exception | 1.count()

    A more accurate exception search using regular expressions

    ([A-Za-z\.]+)Exception | 1.count() 1.exclude(runtime)

    Filter out Runtime exceptions from your search results

    Chart Functions

    chart(line), chart(stacked),chart(cluster), chart(area),chart(scatter)

    Adding a chart command at the end of a search term chooses the visualization type

    chart(pie|table) buckets(1)

    The aggregate chart types require a the bucketskeyword which specifies how many buckets to use. Normally this is set to 1 but can be more if you need it or can be left out completely.