Logscape makes it easy to work with unstructured data. Logscape learns the structure of your data as it is importing it using the Key-Value Discovery feature. This features detects the presence of common valuepairs that exist in widespread formats such as xml and json. A wide range of key value types are supported by Logscape.
It happens automatically without any intervention from the user. However, all fields created in this manner by Logscape will be treated as Strings, if you wish for a field to be treated as a digit, then a Datatype must be created.
Data Discovery happens at index time and is configured on the Data Source for the data. See Data Source Config Page for more.
Any discovered data is listed under the discovered fields and can be searched using the UI or using Logscape Search Syntax directly.
Some applications will log errors wrapped in an XML snippet
Feb 23 18:39:01 occular-sentinel CRON[6249]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime)) < error id="0x00023" host="10.28.1.170" type="Exception"> java.io.FileNotFoundException: fred.txt at java.io.FileInputStream.(FileInputStream.java) at java.io.FileInputStream. (FileInputStream.java) at ExTest.readMyFile(ExTest.java:19) at ExTest.main(ExTest.java:7) </error> Feb 23 18:40:39 occular-sentinel whoopsie[1124]: online
The XML attributes id, host and type will be extracted as system fields and will be searchable.
JSON is growing as an application data exchange format and it is not uncommon to see log files embedding json results along side log messages.
2014-04-12 INFO [requestor-0] Result {
"collection" : {
"title" : "Blog",
"description" : "This is a description of my blog.",
"categories" : [ "Category-1", "Category-2" ]
}
}
The attributes: title, description and categories will be extracted. Collection will not be extracted because its value is another json object.