Class LogFilePatternReceiver
- All Implemented Interfaces:
Plugin
,Component
,org.apache.log4j.spi.OptionHandler
,Thresholdable
- Direct Known Subclasses:
VFSLogFilePatternReceiver
This receiver relies on java.util.regex features to perform the parsing of text in the log file, however the only regular expression field explicitly supported is a glob-style wildcard used to ignore fields in the log file if needed. All other fields are parsed by using the supplied keywords.
Features:
- specify the URL of the log file to be processed
- specify the timestamp format in the file (if one exists, using patterns from SimpleDateFormat
)
- specify the pattern (logFormat) used in the log file using keywords, a wildcard character (*) and fixed text
- 'tail' the file (allows the contents of the file to be continually read and new events processed)
- supports the parsing of multi-line messages and exceptions
- 'hostname' property set to URL host (or 'file' if not available)
- 'application' property set to URL path (or value of fileURL if not available)
- 'group' property can be set to associate multiple log file receivers
Keywords:
TIMESTAMP
LOGGER
LEVEL
THREAD
CLASS
FILE
LINE
METHOD
RELATIVETIME
MESSAGE
NDC
PROP(key)
(NL)
(NL) represents a new line embedded in the log format, supporting log formats whose fields span multiple lines
Use a * to ignore portions of the log format that should be ignored
Example:
If your file's patternlayout is this:
%d %-5p [%t] %C{2} (%F:%L) - %m%n
specify this as the log format:
TIMESTAMP LEVEL [THREAD] CLASS (FILE:LINE) - MESSAGE
To define a PROPERTY field, use PROP(key)
Example:
If you used the RELATIVETIME pattern layout character in the file,
you can use PROP(RELATIVETIME) in the logFormat definition to assign
the RELATIVETIME field as a property on the event.
If your file's patternlayout is this:
%r [%t] %-5p %c %x - %m%n
specify this as the log format:
PROP(RELATIVETIME) [THREAD] LEVEL LOGGER * - MESSAGE
Note the * - it can be used to ignore a single word or sequence of words in the log file (in order for the wildcard to ignore a sequence of words, the text being ignored must be followed by some delimiter, like '-' or '[') - ndc is being ignored in the following example.
Assign a filterExpression in order to only process events which match a filter. If a filterExpression is not assigned, all events are processed.
Limitations:
- no support for the single-line version of throwable supported by patternlayout
(this version of throwable will be included as the last line of the message)
- the relativetime patternLayout character must be set as a property: PROP(RELATIVETIME)
- messages should appear as the last field of the logFormat because the variability in message content
- exceptions are converted if the exception stack trace (other than the first line of the exception)
is stored in the log file with a tab followed by the word 'at' as the first characters in the line
- tailing may fail if the file rolls over.
Example receiver configuration settings (add these as params, specifying a LogFilePatternReceiver 'plugin'):
param: "timestampFormat" value="yyyy-MM-d HH:mm:ss,SSS"
param: "logFormat" value="PROP(RELATIVETIME) [THREAD] LEVEL LOGGER * - MESSAGE"
param: "fileURL" value="file:///c:/events.log"
param: "tailing" value="true"
This configuration will be able to process these sample events:
710 [ Thread-0] DEBUG first.logger first - <test> <test2>something here</test2> <test3 blah=something/> <test4> <test5>something else</test5> </test4></test>
880 [ Thread-2] DEBUG first.logger third - <test> <test2>something here</test2> <test3 blah=something/> <test4> <test5>something else</test5> </test4></test>
880 [ Thread-0] INFO first.logger first - infomsg-0
java.lang.Exception: someexception-first
at Generator2.run(Generator2.java:102)
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate boolean
private static final String
private Map
private String
private SimpleDateFormat
private static final String
private static final String
private final String[]
private static final String
private Pattern
private org.apache.log4j.rule.Rule
private static final String
private String
private String
private static final String
private String
private String
private static final String
private static final String
private int
private String
private static final String
private static final String
private static final String
static final int
private static final String
private static final String
private final String
private static final String
private static final String
private static final String
private String
private static final String
private static final String
private static final String
private Reader
private String
private static final String
private static final String
private Pattern
private boolean
private static final String
private static final String
private String
private String
private boolean
private static final String
private static final String
private long
Fields inherited from class org.apache.log4j.plugins.Receiver
thresholdLevel
Fields inherited from class org.apache.log4j.plugins.PluginSkeleton
active, name
Fields inherited from class org.apache.log4j.spi.ComponentBase
repository
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
Read and process the log file.private org.apache.log4j.spi.LoggingEvent
Construct a logging event from currentMap and additionalLines (additionalLines contains multiple message lines and any exception lines)private String[]
buildException
(int exceptionLine) Combine all exception lines occuring in the additionalLines list into a String arrayprivate String
buildMessage
(String firstMessageLine, int exceptionLine) Combine all message lines occuring in the additionalLines list, adding a newline character between each lineprivate String
Helper method that will convert timestamp format to a patternprivate org.apache.log4j.spi.LoggingEvent
convertToEvent
(Map fieldMap, String[] exception) Convert a keyword-to-values map to a LoggingEventprotected void
private int
Walk the additionalLines list, looking for the EXCEPTION_PATTERN.AccessorAccessorgetGroup()
AccessorAccessorgetPath()
private String
getShortPropertyName
(String longPropertyName) Accessorlong
Accessorprotected void
Build the regular expression needed to parse log entriesboolean
Accessorprivate boolean
boolean
Accessorfinal boolean
When true, this property uses the current Thread to perform the import, otherwise when false (the default), a new Thread is created and started to manage the import.private boolean
passesExpression
(org.apache.log4j.spi.LoggingEvent event) Helper method that supports the evaluation of the expressionprotected void
process
(BufferedReader bufferedReader) Read, parse and optionally tail the log file, converting entries into logging events.private Map
processEvent
(MatchResult result) Convert the match into a map.private String
quoteTimeStampChars
(String input) private String
replaceMetaChars
(String input) Some perl5 characters may occur in the log file format.void
setAppendNonMatches
(boolean appendNonMatches) Mutatorvoid
setCustomLevelDefinitions
(String customLevelDefinitions) If the log file contains non-log4j level strings, they can be mapped to log4j levels using the format (android example): V=TRACE,D=DEBUG,I=INFO,W=WARN,E=ERROR,F=FATAL,S=OFFvoid
setFileURL
(String fileURL) Mutatorvoid
setFilterExpression
(String filterExpression) Mutatorvoid
Mutatorprotected void
void
setLogFormat
(String logFormat) Mutatorprotected void
void
setTailing
(boolean tailing) Mutatorvoid
setTimestampFormat
(String timestampFormat) Mutator.final void
setUseCurrentThread
(boolean useCurrentThread) Sets whether the current Thread or a new Thread is created to perform the import, the default being false (new Thread created).void
setWaitMillis
(long waitMillis) Mutatorvoid
shutdown()
Close the reader.private String
singleReplace
(String inputString, String oldString, String newString) private void
Methods inherited from class org.apache.log4j.plugins.Receiver
doPost, getThreshold, isAsSevereAsThreshold, setThreshold
Methods inherited from class org.apache.log4j.plugins.PluginSkeleton
addPropertyChangeListener, addPropertyChangeListener, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getLoggerRepository, getName, isActive, isEquivalent, removePropertyChangeListener, removePropertyChangeListener, setLoggerRepository, setName
Methods inherited from class org.apache.log4j.spi.ComponentBase
getLogger, getNonFloodingLogger, resetErrorCount
-
Field Details
-
keywords
-
PROP_START
- See Also:
-
PROP_END
- See Also:
-
LOGGER
- See Also:
-
MESSAGE
- See Also:
-
TIMESTAMP
- See Also:
-
NDC
- See Also:
-
LEVEL
- See Also:
-
THREAD
- See Also:
-
CLASS
- See Also:
-
FILE
- See Also:
-
LINE
- See Also:
-
METHOD
- See Also:
-
NEWLINE
- See Also:
-
DEFAULT_HOST
- See Also:
-
EXCEPTION_PATTERN
- See Also:
-
REGEXP_DEFAULT_WILDCARD
- See Also:
-
REGEXP_GREEDY_WILDCARD
- See Also:
-
PATTERN_WILDCARD
- See Also:
-
NOSPACE_GROUP
- See Also:
-
DEFAULT_GROUP
- See Also:
-
GREEDY_GROUP
- See Also:
-
MULTIPLE_SPACES_REGEXP
- See Also:
-
NEWLINE_REGEXP
- See Also:
-
newLine
-
emptyException
-
dateFormat
-
timestampFormat
-
logFormat
-
customLevelDefinitions
-
fileURL
-
host
-
path
-
tailing
private boolean tailing -
filterExpression
-
waitMillis
private long waitMillis -
group
-
VALID_DATEFORMAT_CHARS
- See Also:
-
VALID_DATEFORMAT_CHAR_PATTERN
- See Also:
-
expressionRule
private org.apache.log4j.rule.Rule expressionRule -
currentMap
-
additionalLines
-
matchingKeywords
-
regexp
-
reader
-
regexpPattern
-
exceptionPattern
-
timestampPatternText
-
useCurrentThread
private boolean useCurrentThread -
MISSING_FILE_RETRY_MILLIS
public static final int MISSING_FILE_RETRY_MILLIS- See Also:
-
appendNonMatches
private boolean appendNonMatches -
customLevelDefinitionMap
-
lineCount
private int lineCount
-
-
Constructor Details
-
LogFilePatternReceiver
public LogFilePatternReceiver()
-
-
Method Details
-
getFileURL
Accessor- Returns:
- file URL
-
setFileURL
Mutator- Parameters:
fileURL
-
-
setCustomLevelDefinitions
If the log file contains non-log4j level strings, they can be mapped to log4j levels using the format (android example): V=TRACE,D=DEBUG,I=INFO,W=WARN,E=ERROR,F=FATAL,S=OFF- Parameters:
customLevelDefinitions
- the level definition string
-
getCustomLevelDefinitions
-
isAppendNonMatches
public boolean isAppendNonMatches()Accessor- Returns:
- append non matches
-
setAppendNonMatches
public void setAppendNonMatches(boolean appendNonMatches) Mutator- Parameters:
appendNonMatches
-
-
getFilterExpression
Accessor- Returns:
- filter expression
-
setFilterExpression
Mutator- Parameters:
filterExpression
-
-
isTailing
public boolean isTailing()Accessor- Returns:
- tailing
-
setTailing
public void setTailing(boolean tailing) Mutator- Parameters:
tailing
-
-
isUseCurrentThread
public final boolean isUseCurrentThread()When true, this property uses the current Thread to perform the import, otherwise when false (the default), a new Thread is created and started to manage the import.- Returns:
-
setUseCurrentThread
public final void setUseCurrentThread(boolean useCurrentThread) Sets whether the current Thread or a new Thread is created to perform the import, the default being false (new Thread created).- Parameters:
useCurrentThread
-
-
getLogFormat
Accessor- Returns:
- log format
-
setLogFormat
Mutator- Parameters:
logFormat
- the format
-
setGroup
Mutator -
getGroup
Accessor- Returns:
- group
-
setTimestampFormat
Mutator. Specify a pattern fromSimpleDateFormat
- Parameters:
timestampFormat
-
-
getTimestampFormat
Accessor- Returns:
- timestamp format
-
getWaitMillis
public long getWaitMillis()Accessor- Returns:
- millis between retrieves of content
-
setWaitMillis
public void setWaitMillis(long waitMillis) Mutator- Parameters:
waitMillis
-
-
getExceptionLine
private int getExceptionLine()Walk the additionalLines list, looking for the EXCEPTION_PATTERN.Return the index of the first matched line (the match may be the 1st line of an exception)
Assumptions:
- the additionalLines list may contain both message and exception lines
- message lines are added to the additionalLines list and then exception lines (all message lines occur in the list prior to all exception lines)- Returns:
- -1 if no exception line exists, line number otherwise
-
buildMessage
Combine all message lines occuring in the additionalLines list, adding a newline character between each linethe event will already have a message - combine this message with the message lines in the additionalLines list (all entries prior to the exceptionLine index)
- Parameters:
firstMessageLine
- primary message lineexceptionLine
- index of first exception line- Returns:
- message
-
buildException
Combine all exception lines occuring in the additionalLines list into a String array(all entries equal to or greater than the exceptionLine index)
- Parameters:
exceptionLine
- index of first exception line- Returns:
- exception
-
buildEvent
private org.apache.log4j.spi.LoggingEvent buildEvent()Construct a logging event from currentMap and additionalLines (additionalLines contains multiple message lines and any exception lines)CurrentMap and additionalLines are cleared in the process
- Returns:
- event
-
process
Read, parse and optionally tail the log file, converting entries into logging events.A runtimeException is thrown if the logFormat pattern is malformed.
- Parameters:
bufferedReader
-- Throws:
IOException
-
createPattern
protected void createPattern() -
passesExpression
private boolean passesExpression(org.apache.log4j.spi.LoggingEvent event) Helper method that supports the evaluation of the expression- Parameters:
event
-- Returns:
- true if expression isn't set, or the result of the evaluation otherwise
-
processEvent
Convert the match into a map.Relies on the fact that the matchingKeywords list is in the same order as the groups in the regular expression
- Parameters:
result
-- Returns:
- map
-
convertTimestamp
Helper method that will convert timestamp format to a pattern- Returns:
- string
-
setHost
-
setPath
-
getPath
-
initialize
protected void initialize()Build the regular expression needed to parse log entries -
updateCustomLevelDefinitionMap
private void updateCustomLevelDefinitionMap() -
isInteger
-
quoteTimeStampChars
-
singleReplace
-
getShortPropertyName
-
replaceMetaChars
Some perl5 characters may occur in the log file format. Escape these characters to prevent parsing errors.- Parameters:
input
-- Returns:
- string
-
convertToEvent
Convert a keyword-to-values map to a LoggingEvent- Parameters:
fieldMap
-exception
-- Returns:
- logging event
-
shutdown
public void shutdown()Close the reader. -
activateOptions
public void activateOptions()Read and process the log file.
-