Feature-request: remove record from sincedb if file was deleted

[ov7a]

Motivation:
If multiple files are processed, it seems what record is left in since_db. If the record with be removed since_db will occupy less space, which would be nice if a lot of files are processed.

[guyboertje]

@ov7a - Thanks for your feature request.

Some form of this feature is already implemented in pull request #79. This feature is aligned somewhat with the Filebeat handling of the same issue. When we update the sincedb record we change the last_seen_at timestamp. Then periodically we will sweep through the sincedb records and remove them if they were last seen before than a user defined time ago. We need to do it this way because a file can be reintroduced and we would have no record of it. The user should decide what a reasonable timeframe for retention.

Unfortunately the PR is delayed because of its complexity and other team duties.