Zlib._handle.onerror unexpected end of file

[Brideau]

I am receiving the same error as mentioned in issue #269, but the solution mentioned there does not fix the problem for me.

I'm using twit version 2.2.4, with the following configuration:

const keys = {
    consumer_key : process.env.TWITTER_CONSUMER_KEY,
    consumer_secret : process.env.TWITTER_CONSUMER_SECRET,
    access_token : process.env.TWITTER_ACCESS_TOKEN,
    access_token_secret : process.env.TWITTER_ACCESS_TOKEN_SECRET,
    timeout_ms: 60*1000,
  };

I've tried it both with and without the trailing comma at the end. It works for an extended period of time (usually a day or two), but then fails abruptly with this error:

events.js:160
    throw er; // Unhandled 'error' event
    ^

Error: unexpected end of file
    at Zlib._handle.onerror (zlib.js:370:17)

The tweet emitter is started and consumed using the following functions (I was testing different geographic areas, hence the geo filter that covers the whole planet):

const client = new Twitter(keys);
const twitEmitter = client.stream('statuses/filter',
  {locations: '-180,-89,180,89'});

twitEmitter.on('tweet', (tweet) => {
  filterForwardLog(tweet);
});

twitEmitter.on('disconnect', (disconnectMessage) => {
  winston.log('warning', 'Connection terminated: ');
  reject(disconnectMessage);
});

twitEmitter.on('connect', () => {
  winston.log('info', 'Connection attempted.');
});

twitEmitter.on('connected', () => {
  winston.log('info', 'Connection successful.');
});

twitEmitter.on('reconnect', (req, res, interval) => {
  winston.log('notice', `Reconnecting in ${interval / 1000.0} seconds`);
});

twitEmitter.on('warning', (warning) => {
  winston.log('warn', warning);
});

[olalonde]

Same thing here. This bug alone accounts for like 90%+ of my error logs, pretty annoying.

{ Error: unexpected end of file
    at Zlib._handle.onerror (zlib.js:370:17) errno: -5, code: 'Z_BUF_ERROR' }
error:  Error: unexpected end of file
    at Zlib._handle.onerror (zlib.js:370:17)

[olalonde]

Created an issue on node (nodejs/node#8701) just in case. I reviewed the twit code and it doesn't seem to do anything out of ordinary with the zlib streams. And if it was just an error that was thrown when a connection closes early or timeout, I suppose googling for that error would bring a lot more results (but it doesn't).

[chasset]

Hello, I have the same Z_BUF_ERROR. I would like to share a clue of the reason and the solution:

Twitter sends tweets in chunks. Sometimes, some chunks get lost. This throws an error when zlib tries to look for an end of the tweet. A correct behaviour for Twit would be to stay connected and to wait for the next good tweet.

I discover this developing in Perl my own library to do what Twit does.

What do you think about it?

[hoodsy]

Anyone have a temporary workaround or fix? This is basically shutting down my application.

I haven't recently changed versions of node or twit, but this error has randomly come around...

[ioRekz]

Yes same here, started since few days

[hoodsy]

@ioRekz do you remember changing anything around the time this happened? Could this be a change on Twitter's end with the streaming API?

[CTassisF]

This error always existed for me and it was quite intermittent, but today it got worse. Twitter Streaming API is closing too much connections today (maybe due to DNS problems they had with Dyn), you can see that yourself using cURL to stream (the TCP session should disconnect in few seconds).
The problem with ttezel/twit is that it can't handle connections being abruptly closed by Twitter streaming endpoint. Instead of crashing, ttezel/twit needs to be able to handle these events and try to reconnect streams incrementally, but I do not know how to do that :/

[hoodsy]

@CTassisF my connection is polling, so it will reconnect after a short period of time. Must be related to the DNS attack on Dyn, as you say.

[CTassisF]

This issue might be related to NodeJS. I was using NodeJS 6.9.1 from nodesource/distributions but I decided to abandon it and try to use the NodeJS 4.6.0 from Debian Stretch official mirror. With NodeJS 4.6.0 the issue is gone.

[austinhuang0131]

Same here, started today.

[olalonde]

@CTassisF I tried opening an issue nodejs/node#8701 but didn't have time to write some code to reproduce. I'm using Node v6. Maybe #304 will magically fix the bug? 😜

[ioRekz]

Who is using heroku ? I don't seem to have any error when starting the server locally.

[ioRekz]

@hoodsy Ho thought I answered but no, I'm sure I didn't change anything

[ioRekz]

@CTassisF I don't have the same behavior. It's almost every favorite and track events in my case that raise this error

[olalonde]

Not Heroku but Docker