How to remove documents during incoming/outgoing handlers?

[JamesMGreene]

I'm looking to setup a PouchDB environment such that documents can contain an expiry property (e.g. an expiresAt timestamp).

As such, I am interested in if I can utilize this plugin's incoming/outgoing functionality to check a document's expiry when being read/written and verify it should still exist. If so, return it; if not, remove it from the database and do not return it.

Looking through the code and issue tracker, it is not at all clear how to achieve this.

The README does, however, mention this basic concept in passing:

This allows you to:

• ...
• Remove or modify documents before storage (e.g. to massage data from CouchDB)

Can someone please help me understand how to hook such a thing up?

[gr2m]

try this example from the README to delete a doc when reading out from the db if it’s expired and throwing the missing error

var errors = require('pouchdb-errors')
var pouch = new PouchDB('mydb');
pouch.transform({
  outgoing: function (doc) {
    if (doc.expiresAt < now) {
      pouch.remove(doc).then(() => throw errors.MISSING_DOC)
    }
    return return doc
  }
});

hope that helps to point you in the right direction?

[JamesMGreene]

try this example from the README

Huh? I definitely do not see this example in the README.

That said, your example generally makes sense to me. So throwing a errors.MISSING_DOC Error will not prevent the rest of a documents from being returned if it is a batch result? The ideal behavior I'm seeking would be to remove the document transparently and still allow the rest of the result set to return without missing result indices, errors, etc.

[gr2m]

I don’t know, you’ll have to experiment with it. Would be great if you could share what you came up with

[JamesMGreene]

So I've tried a number of different variations on the idea from @gr2m's comment above but they all suffer from the same problem: the document isn't fully removed until after the query/read resolves, and this transform-pouch plugin doesn't allow our handlers access to the full result document, nor the top-level result object (e.g. from allDocs), in order to modify them in a fake fashion manually.

Experiments

Experiment 1: Remove doc and throw

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      return db
        .remove( doc )
        .then(function() {
          throw errors.MISSING_DOC;
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the whole call results in the errors.MISSING_DOC error, so _NO _ results are returned. This does not match my desired behavior.

The next allDocs query does return the expected results as desired (unless another document expires and blocks the result from returning as well, that is).

Experiment 2: Update doc with _deleted: true and throw

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      doc._deleted = true;
      return db
        .put( doc )
        .then(function() {
          throw errors.MISSING_DOC;
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the whole call results in the errors.MISSING_DOC error, so _NO _ results are returned.

The next allDocs query does return the expected results as desired (unless another document expires and blocks the result from returning as well, that is).

Experiment 3: Remove doc and return undefined

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      return db
        .remove( doc )
        .then(function() {
          return;  // undefined
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the document is still returned in the results. However, its contents (a) does NOT have a value.deleted property set to true, and (b) no longer has a doc property value. This does not match my desired behavior.

The next allDocs query does return the expected results as desired.

Same behavior occurs when doing return null/return {} instead, only the doc property value becomes null/{}, respectively.

Experiment 4: Update doc with _deleted: true and return undefined

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      doc._deleted = true;
      return db
        .put( doc )
        .then(function() {
          return; // undefined
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the document is still returned in the results. However, its contents (a) does NOT have a value.deleted property set to true, and (b) no longer has a doc property value. This does not match my desired behavior.

The next allDocs query does return the expected results as desired.

Same behavior occurs when doing return null/return {} instead, only the doc property value becomes null/{}, respectively.

Experiment 5: Remove doc and return doc

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      return db
        .remove( doc )
        .then(function() {
          return doc;
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the document is still returned in the results. However, its contents (a) does NOT have a value.deleted property set to true, and (b) its doc property is still fully intact. This does not match my desired behavior.

The next allDocs query does return the expected results as desired.

Experiment 6: Update doc with _deleted: true and return doc

Code:

db.transform({
  outgoing: function( doc ) {
    if ( doc.expiresAt < Date.now() ) {
      doc._deleted = true;
      return db
        .put( doc )
        .then(function() {
          return doc;
        });
    }
    return doc;
  }
});

Result:

Using allDocs, when a given document in the result set becomes expired, the document is still returned in the results. However, its contents (a) does NOT have a value.deleted property set to true, (b) has a doc._deleted property set to true, and (c) the rest of its doc property is still fully intact. This does not match my desired behavior.

The next allDocs query does return the expected results as desired.

Conclusion

If attempting to use this plugin as it is today to do this work, I would still have to add a post-operation outside of the transform to then filter the results as well (e.g. omit all documents where doc is not defined after an allDocs call). 😞