Skip to content

Buffering the input stream for metadata read#160

Open
Yaliang wants to merge 442 commits into
twitter-forks:twitter-masterfrom
Yaliang:yaliangw/twitter/parquet-in-buffer
Open

Buffering the input stream for metadata read#160
Yaliang wants to merge 442 commits into
twitter-forks:twitter-masterfrom
Yaliang:yaliangw/twitter/parquet-in-buffer

Conversation

@Yaliang
Copy link
Copy Markdown

@Yaliang Yaliang commented Jul 5, 2018

No description provided.

Yaliang and others added 30 commits December 9, 2016 11:55
…seVersion

Update release version to 0.157-tw-0.29
…alidate

invoke isReachable to test connection
warn instead of fail when JVM is non-Oracle
This reverts commit 5dd9eda, reversing
changes made to 8d4e86a.
…radeVersion"

This reverts commit 8d4e86a, reversing
changes made to 76f4364.
…nectionValidate"

This reverts commit 76f4364, reversing
changes made to 0cf760d.
Yaliang and others added 22 commits March 20, 2018 19:15
Add utility functions for snowflake ids (https://developer.twitter.com/en/docs/basics/twitter-ids).

is_snowflake(id) → boolean
Return if a bigint is a snowflake ID (true/false).

first_snowflake_for(timestamp) → bigint
Return the first snowflake ID given a timestamp.

timestamp_from_snowflake(id) → timestamp
Return the timestamp given a snowflake ID.

cluster_id_from_snowflake(id) → bigint
Return the cluster ID given a snowflake ID.

instance_id_from_snowflake(id) → bigint
Return the instance ID given a snowflake ID.

sequence_num_from_snowflake(id) → bigint
Return the sequence number given a snowflake ID.
Upgrade to 0.196 based version
…witter-forks#155)

According to Parquet spec:
If the repeated field is a group with one field and is named either array
or uses the LIST-annotated group's name with _tuple appended
then the repeated type is the element type and elements are required.

https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists

(cherry picked from commit 346f38e)
This fixes prestodb#10383, where deployments using the legacy resource group
configuration manager were unable to run more than a single query that
used memory concurrently.
@Yaliang Yaliang force-pushed the yaliangw/twitter/parquet-in-buffer branch from 6b8f866 to 01812d5 Compare July 5, 2018 20:10
@Yaliang Yaliang force-pushed the yaliangw/twitter/parquet-in-buffer branch from 855a451 to 5a47548 Compare February 28, 2019 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants