Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -113,3 +113,18 @@ venv.bak/

# PyCharm
.idea

# Google credentials — NEVER commit these
client_secrets.json
client_secret*.json
credentials.json
settings.yaml
token.json
token.pickle
mycreds.txt
*-key.json
*_key.json
*service*account*.json
*serviceaccount*.json
sa-key*.json
.credentials/
116 changes: 111 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
## Features

- 🚀 **Simple public downloads** - No API setup required for public files
- 🔒 **Private files too** - Pass an authenticated `drive=` to reach files shared only with you
- 🗂️ **Files as dict** - `GDFiles` keyed by file id *or* Drive URL
- 📂 **Folder as dict** - Browse folders with `dict`-like interface
- 💾 **Read/write operations** - Full CRUD support through mapping protocol
- 🔄 **Recursive traversal** - Control depth with `max_levels`
Expand Down Expand Up @@ -55,6 +57,62 @@ get_bytes(url, local_path="/path/to/save.pdf")
get_bytes(url, use_cache=True) # Uses ~/.cache/pydrivedol/cached/
```

If the file is **not** publicly shared, Google answers with its HTML sign-in page — under
HTTP 200, so it looks like a successful download. pydrivedol refuses to hand that back as
file content and raises `NotPubliclyShared` instead, telling you to authenticate:

```python
from pydrivedol import get_bytes, NotPubliclyShared

try:
content = get_bytes(url)
except NotPubliclyShared:
content = get_bytes(url, drive=drive) # see below
```

(If you really are downloading an HTML file, pass `allow_html=True`.)

### Private Files (Requires API Setup)

Authenticate once, then pass the `drive` wherever bytes are needed:

```python
from pydrivedol import drive_from_service_account, get_bytes, get_metadata, GDFiles

drive = drive_from_service_account("service-account-key.json")

# Is it worth downloading? Metadata is cheap — it fetches no content.
info = get_metadata(url, drive=drive)
info["title"], info["fileSize"], info["mimeType"], info["modifiedDate"]
# ('client_export.xlsx', 18512345, 'application/vnd...sheet', '2026-08-01T12:00:00.000Z')

# Same get_bytes, now authenticated — local_path= and use_cache= work as before
content = get_bytes(url, drive=drive)
```

`fileSize` comes back as an `int` (Drive sends it as a string), and is **absent** for
Google-native files — Sheets/Docs/Slides have no stored byte size.

### Files as a Mapping: `GDFiles`

When files arrive as *links* rather than as a folder listing, key by the link:

```python
files = GDFiles(drive)

content = files[url] # a Drive file URL...
content = files[file_id] # ...or the bare id: same entry
url in files # metadata probe, never a download
files.metadata(url) # name / size / mimeType / modifiedDate

# Iteration needs a scope — an unscoped GDFiles addresses the whole Drive
scoped = GDFiles(drive, folder_url=folder_url)
list(scoped) # file ids
```

Unscoped, `iter()` and `len()` raise `NotImplementedError` (with a message naming
`folder_url=`) rather than silently paginating your entire Drive. Lookup works either way.

### Working with Folders (Requires API Setup)

```python
Expand Down Expand Up @@ -151,6 +209,40 @@ reader = GDReader(folder_url)

That's it! You only need to authenticate once.

### Headless / server: use a service account instead

The browser flow above is unusable from a script, a server, or an agent. For those, create a
**service account** — a robot identity with its own key file and no interactive login:

1. [Google Cloud Console](https://console.cloud.google.com/) → your project →
"APIs & Services" → "Credentials"
2. "Create Credentials" → **Service account**. Any name; no roles needed.
3. Open the new service account → "Keys" → "Add key" → "Create new key" → **JSON**. The key
file downloads once and cannot be re-downloaded.
4. Copy the service account's **`client_email`** (it looks like
`something@your-project.iam.gserviceaccount.com`).
5. In Google Drive, **share the file or folder with that `client_email`** — Viewer to read,
Editor to write. This is the step people forget: the service account is a separate
identity, and a file shared with *you* is not shared with *it*.
6. Point pydrivedol at the key file:

```python
from pydrivedol import drive_from_service_account

drive = drive_from_service_account("service-account-key.json")
```

Keep the key file out of version control (this repo's `.gitignore` covers the usual names) and
out of the repo entirely if you can — read its path from an environment variable.

Read-only by token, if you want the extra guarantee:

```python
drive = drive_from_service_account(
key_file, scopes=("https://www.googleapis.com/auth/drive.readonly",)
)
```

## Advanced Usage

### Control Recursion Depth
Expand Down Expand Up @@ -281,11 +373,14 @@ pydrivedol follows the `dol` package patterns:

```
Helper Functions
└─ get_bytes() # Simple public downloads (no API)

API-Based Classes
└─ GDReader (Mapping) # Read-only folder access
└─ GDStore (MutableMapping) # Read-write folder access
└─ get_bytes(url) # Simple public downloads (no API)
└─ get_bytes(url, drive=...) # Authenticated: reaches private files
└─ get_metadata(url, drive=...) # Name/size/mimeType/date, no download

API-Based Classes
└─ GDFiles (Mapping) # Files, keyed by file id or file URL
└─ GDReader (Mapping) # Read-only folder access, keyed by path
└─ GDStore (MutableMapping) # Read-write folder access
```

**Design Principles:**
Expand Down Expand Up @@ -410,6 +505,17 @@ pip install pydrive2
2. Verify folder sharing settings
3. Re-authenticate: delete saved credentials and run again

### `NotPubliclyShared` from `get_bytes`

The file is not shared "anyone with the link", so the unauthenticated endpoint cannot reach
it. Either make it public, or authenticate and pass `drive=` (see *Private Files* above). For
a service account, remember the file/folder must be shared with the account's `client_email`.

### Downloaded file won't open / `BadZipFile` on an xlsx

You are almost certainly holding Google's sign-in page rather than the file. Recent versions
raise `NotPubliclyShared` for exactly this; if you are pinned to an older one, upgrade.

### Tests are skipped

Check environment variables:
Expand Down
24 changes: 22 additions & 2 deletions pydrivedol/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@

Key Features:
- get_bytes(url): Download bytes from public Google Drive URLs (no API setup needed)
- get_bytes(url, drive=...): Same, but authenticated -- reaches *private* files
- get_metadata(url, drive=...): Name/size/mimeType/modifiedDate without downloading
- GDFiles: Read-only Mapping of files, keyed by file id or file URL
- GDReader: Read-only Mapping interface to a Google Drive folder
- GDStore: Read-write MutableMapping interface to a Google Drive folder

Expand All @@ -17,11 +20,24 @@

Usage:

Simple download (no API needed)
Simple download (no API needed) -- public files only. If the file is not public,
Google answers with its HTML sign-in page (under HTTP 200); pydrivedol raises
NotPubliclyShared rather than handing that page back as the file's content.

>>> content = get_bytes(url) # doctest: +SKIP
>>> path = get_bytes(url, local_path=True) # doctest: +SKIP

Private files: authenticate once, then pass the drive around

>>> drive = drive_from_service_account('service-account-key.json') # doctest: +SKIP
>>> get_metadata(url, drive=drive)['fileSize'] # decide before downloading # doctest: +SKIP
>>> content = get_bytes(url, drive=drive) # doctest: +SKIP

Files as a Mapping, keyed by file id or URL

>>> files = GDFiles(drive) # doctest: +SKIP
>>> content = files[url] # doctest: +SKIP

Read folder

>>> reader = GDReader(folder_url) # doctest: +SKIP
Expand All @@ -43,8 +59,12 @@
"""

from pydrivedol.base import (
get_bytes, # simple download function
get_bytes, # simple download function (authenticated when given drive=)
NotPubliclyShared, # raised when a public download returns Google's sign-in page
# Needing API setup:
get_metadata, # a file's name/size/mimeType/modifiedDate, without downloading it
DEFAULT_METADATA_FIELDS, # the metadata fields get_metadata asks for by default
GDFiles, # read-only Mapping of files, keyed by file id or file URL
GDReader, # read-only Mapping interface to a Google Drive folder
GDStore, # read-write MutableMapping interface to a Google Drive folder
drive_from_service_account, # headless auth: build a GoogleDrive from an SA key
Expand Down
Loading
Loading