Define threshold condition on Elasticsearch data

[ehsanjavadynia]

Hi, I've read the elasticsearch alerter document. but I didn't realize how I can define my condition based on source.
our elasticsearch document is something like this:
{ "key": "my-key", "value": 20 }

I want to define an alert which detect value higher than 45. Is it possible in hawkular?
I've already tried mapping value to text (using mapping field in context) and it didn't work. value won't be mapped to text.
is it possible to write a condition like this:
"conditions": [ { "type": "EVENT", "dataId": "myDataId", "expression": "source.value > 20" } ]
and can we use other condition types with elasticsearch plugin? (threshold, compare, ...).
Could you help me with this? I have spent two days reading documents and trying...
Thanks in advance

[lucasponce]

Hello @ehsanjavadynia,
First, thank you for evaluating Hawkular Alerting.
The scenario you comment should be possible, there is a complete example here we can use to modify to your model:
https://github.com/hawkular/hawkular-alerts/tree/master/examples/elasticsearch
Feel free to take a look into it, and modify it to get familiar with the API.
Basically it expects Elasticsearch documents with the format:

{
    "@timestamp":"...",
    "level":"...",
    "app":"...",
    "message":"..."
}

https://github.com/hawkular/hawkular-alerts/blob/master/examples/elasticsearch/create-logs.sh

The Elasticsearch alerter transform an Elasticsearch document into a Hawkular Event, and this requires a transformation to define which Elasticsearch field goes to target Hawkular Event, in the example this is defined in the trigger context mapping field like

"mapping": "level:category,@timestamp:ctime,message:text,app:dataId,index:tags"

https://github.com/hawkular/hawkular-alerts/blob/master/examples/elasticsearch/elasticsearch-triggers.json#L18

Once the mapping is in place, the Elasticsearch alerter will pull ES documents and feed Hawkular Events that are ready to be evaluated with Event conditions like

{
   "type": "EVENT",
   "dataId": "AppA",
   "expression": "category == 'ERROR',text contains 'Backend'"
}

https://github.com/hawkular/hawkular-alerts/blob/master/examples/elasticsearch/elasticsearch-triggers.json#L35

So, it is important to note that any event/data in Hawkular needs an id as a source to refer in the condition, so, that normally should be defined in the mapping from Elasticsearch -> Hawkular.

So, in your example, I would add a "@timestamp" data in the Elasticsearch document, that will help Kibana and Hawkular to have a time series data, something like:

{
   "@timestamp":"...",  // This shoud be provided by the writer of this document in ES
   "key": "my-key",
   "value":"20"
}

So, in the trigger mapping we could then define something like:

"mapping": "'@timestamp:ctime,value:text,key:dataId"

And then define your condition like:

{
   "type": "EVENT",
   "dataId": "my-key",
   "expression": "text > 20"
}

In theory, it should work, the Hawkular Event is thought for complex documents (which is the normal Elasticsearch use cases we have been working on, like logs info or similar) and perhaps for this specific example mapping into a simple Hawkular Data may work better, that is something we can plan in the roadmap, but using Event should work for your scenario.

Please, let me know if these examples and steps work for your scenario.

Lucas

[ehsanjavadynia]

Thank you @lucasponce, your comment was very helpful.
I've followed elasticsearch example, but I think there is problem with elasticsearch alerter plugin, it won't map values that are not instances of string to hawkular event fields:
https://github.com/hawkular/hawkular-alerts/blob/master/alerters/alerters-plugins/alerters-elasticsearch/src/main/java/org/hawkular/alerter/elasticsearch/ElasticsearchQuery.java#L395

I've added number to this function:

protected String getField(Map<String, Object> source, String name) {
        if (source == null || name == null) {
            return null;
        }
        if (name.charAt(0) == '\'' && name.charAt(name.length() - 1) == '\'') {
            return name.substring(1, name.length() - 1);
        }
        String[] names = name.split("\\|");
        String defaultValue = "";
        if (names.length > 1) {
            if (names[1].charAt(0) == '\'' && names[1].charAt(names[1].length() - 1) == '\'') {
                defaultValue = names[1].substring(1, names[1].length() - 1);
            }
            name = names[0];
        }
        String[] fields = name.split("\\.");
        for (int i=0; i < fields.length; i++) {
            Object value = source.get(fields[i]);
            if (value instanceof String) {
                return (String) value;
            }
            if (value instanceof Number) {
                return String.valueOf(value);
            }
            if (value instanceof Map) {
                source = (Map<String, Object>) value;
            }
        }
        return defaultValue;
    }

we couldn't run alert engine without docker. so I compiled the code and pushed the jar file into docker. now it's working.
Thanks for your great work, it's really impressive 👍

[lucasponce]

ah, good catch !
Do you mind to send a PR to master ?
I can open a JIRA on our system and link it to your PR
https://issues.jboss.org/projects/HWKALERTS/issues/

Really appreciate you can use this and help us to improve it.
Thx.

[lucasponce]

I have created
https://issues.jboss.org/browse/HWKALERTS-286
Feel free is you want to join to our system to take the ownership to mark that you are the author of the fix or I can link the PR there.
Thx.

[ehsanjavadynia]

Sure! thanks, I've joined jira, and I've created a pull request here:
#478