Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.5.0
current_version = 0.6.0
commit = true
tag = true
tag_name = v{new_version}
Expand Down
147 changes: 147 additions & 0 deletions docs/updates-with-mutations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Pydongo Mutation System Guide

## Updates from Mutations

Pydongo's mutation system allows you to perform atomic MongoDB updates on large numbers of documents without loading them into memory.

Instead of fetching documents, modifying them in Python, and saving them one by one (which becomes slow and memory-intensive for thousands or millions of records), you can compose MongoDB update operations directly using familiar Python syntax.

This is the primary intended use case of the Mutation DSL.

## Core Idea

* **Write Python-like syntax -> Generate efficient MongoDB bulk/atomic updates**
* **No documents are loaded into memory when using collection-level mutations**

### Example

```python
# Efficient: updates 10,000+ documents in one MongoDB operation
query = users.find(users.followers < 50)
query.followers += 10
query.posts.add_to_set(Post(content="Thanks for being active!"))
query.mutate() # Single updateMany() call
```

## How to Create and Apply Mutations

Mutations are collected automatically inside a context created when you call `.find()`.

### Recommended Bulk Update Pattern

```python
query = collection.find(collection.age > 30)

# These statements build mutations in the shared context
query.status = "active" # $set
query.score += 100 # $inc
query.tags.add_to_set("vip") # $addToSet
query.profile.views.setmax(1000) # $max
query.bio.unset() # $unset

# Execute the bulk update (updateMany)
result = query.mutate() # or await query.amutate()
```

You can always inspect the generated update document:

```python
print(query.get_mutations())
# {
# "$set": {"status": "active"},
# "$inc": {"score": 100},
# "$addToSet": {"tags": "vip"},
# "$max": {"profile.views": 1000},
# "$unset": {"bio": ""}
# }
```

### Important Warning: Shared Mutation Context

The mutation context is thread-local / event-loop-local and shared across all `.find()` calls in the same execution context.

#### Danger Example

```python
# DANGER - same context is reused!
q1 = users.find(users.country == "FR")
q1.followers += 5

q2 = users.find(users.country == "DE") # ← same mutation context!
q2.followers += 10

q1.mutate() # Will apply BOTH +5 and +10 to French users!
```

### Recommended Safe Patterns

**Pattern 1: Short-lived query builder**

```python
with users.find(users.age > 65) as elderly:
elderly.status = "retired"
elderly.pension += 200
elderly.mutate()
```

**Pattern 2: Explicit reset**

```python
query = users.find(users.active == True)
query.last_seen = datetime.utcnow()
query.mutate()
MutationExpressionContext.clear() # Reset for next query
```

> In most real applications, prefer short-lived query builders.

## Supported Mutation Operations

| Python Syntax | MongoDB Operator | Example / Notes |
| --------------------------- | ---------------- | ------------------------------ |
| field = value | $set | user.name = "Alice" |
| field.unset() | $unset | Removes field completely |
| numeric += n / numeric -= n | $inc | followers += 10, balance -= 50 |
| numeric *= n | $mul | score *= 1.5 |
| numeric /= n | $mul (1/n) | price /= 2 |
| field.setmax(value) | $max | Keeps greater value |
| field.setmin(value) | $min | Keeps smaller value |
| array.push(value) | $push | Append value |
| array.add_to_set(value) | $addToSet | Add only if not present |
| array.pull(value) | $pull | Remove matching values |
| array.popleft() | $pop: 1 | Remove first element |
| array.popright() | $pop: -1 | Remove last element |
| nested.field = value | dot notation | user.address.city = "Paris" |

## Single Document Updates (less common use case)

When you already have a document loaded (`find_one` / `afind_one`), you can still use mutation syntax, but the traditional approach is usually simpler:

```python
user = await users.afind_one(users.username == "alice")

user.bio = "Loves async Python"
user.followers += 1
user.posts.push(Post(content="New post!"))

await user.save() # Applies mutations + full document save
```

> For single documents, modifying the Python object and calling `.save()` is usually more readable.

The real power of the Mutation DSL appears when performing bulk updates without fetching documents.

## When to Use Which Approach

| Goal | Recommended Approach | Memory Usage | Speed (large N) |
| -------------------------------------- | ------------------------------ | ------------ | --------------- |
| Update 1 document (already loaded) | Modify object -> .save() | Low | - |
| Update 1 document (without loading) | .find_one(...).mutate() | None | Fast |
| Update hundreds / thousands / millions | .find(...).mutate() | None | Very fast |
| Complex per-document logic | Load -> process -> save (loop) | High | Slow |

Pydongo's Mutation DSL lets you harness MongoDB's atomic `updateMany` power using clean, type-safe Python syntax.

Perfect for bulk maintenance tasks, counter updates, user rewards, data cleanups, and migrations.

See the `social_media_app.py` example for a complete demonstration of rewarding many users in a single operation.
147 changes: 100 additions & 47 deletions examples/async/social_media_app.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
import asyncio
import uuid
from typing import Union
from typing import Optional, List

from pydantic import BaseModel
from pydantic import Field
from pydantic import BaseModel, Field

from pydongo import as_collection
from pydongo import as_document
from pydongo import as_collection, as_document
from pydongo.drivers.async_mongo import PyMongoAsyncDriver

# === MODELS ===

# =========================
# Models
# =========================

class Post(BaseModel):
id: str = Field(default_factory=lambda: str(uuid.uuid4()))
Expand All @@ -21,90 +21,143 @@ class Post(BaseModel):
class User(BaseModel):
username: str
email: str
bio: Union[str, None] = ""
posts: list[Post] = []
bio: Optional[str] = ""
followers: int = 0

posts: List[Post] = Field(default_factory=list)
pinned_post: Optional[Post] = None

# === DB SETUP ===

driver = PyMongoAsyncDriver("mongodb://localhost:27017", "social_app_async")
# =========================
# Database setup
# =========================

driver = PyMongoAsyncDriver(
connection_string="mongodb://localhost:27017",
database_name="social_app_async",
)

# === OPERATIONS ===
users = as_collection(User, driver)


# =========================
# Async CRUD operations
# =========================

async def create_user(username: str, email: str, bio: str = "") -> User:
user = User(username=username, email=email, bio=bio)
user = User(
username=username,
email=email,
bio=bio,
pinned_post=Post(content="Welcome to my profile"),
)
doc = as_document(user, driver)
await doc.save()
return doc


async def make_post(username: str, content: str) -> Union[Post, None]:
users = as_collection(User, driver)
user_doc = await users.find_one(users.username == username)
if not user_doc:
async def make_post(username: str, content: str) -> Optional[Post]:
user = await users.afind_one(users.username == username)
if not user:
return None

post = Post(content=content)
user_doc.posts.append(post)
await user_doc.save()
user.posts.append(post)
await user.save()
return post


async def like_post(username: str, post_id: str) -> bool:
users = as_collection(User, driver)
user_doc = await users.find_one(users.username == username)
if not user_doc:
user = await users.afind_one(users.username == username)
if not user:
return False

for post in user_doc.posts:
for post in user.posts:
if post.id == post_id:
post.likes += 1
await user_doc.save()
await user.save()
return True

return False


async def get_all_posts(username: Union[str, None] = None) -> list[Post]:
users = as_collection(User, driver)
# =========================
# Mutation-based updates
# =========================

if username:
user_doc = await users.find_one(users.username == username)
return user_doc.posts if user_doc else []
async def reward_active_users(bonus: int) -> None:
"""
Perform mutation-based updates on users with fewer than 50 followers.
Demonstrates:
- numeric mutation ($inc)
- nested field mutation
- array mutation ($addToSet)
"""
query = users.find(users.followers < 50)

all_users = await users.find().all()
all_posts = []
for user in all_users:
all_posts.extend(user.posts)
return all_posts
# Numeric mutation
query.followers += bonus

# Nested field mutation
if query.pinned_post is not None:
query.pinned_post.likes += bonus

# Array mutation
query.posts.add_to_set(Post(content="Thanks for being active!"))

result = await query.amutate()

print("Mutation result:", result)
print("Applied mutations:", query.get_mutations())

# === DEMO FLOW ===

# =========================
# Demo flow
# =========================

async def main() -> None:
await driver.connect()

# Create a user
sam = await create_user("samdev", "sam@example.com", bio="Async all the way!")
print("Created user:", sam.username)
# Create users
alice = await create_user("alice", "alice@example.com", bio="Loves async")
bob = await create_user("bob", "bob@example.com", bio="Async explorer")

# Make a post
post = await make_post("samdev", "Async + Mongo + Pydongo 😎")
print("Posted:", post.content)
# Make posts
await make_post("alice", "Hello Async World!")
await make_post("bob", "Python + Mongo + Async Rocks!")

# Like the post
liked = await like_post("samdev", post.id)
print("Liked post:", "✅" if liked else "❌")
# Like posts
for user in [alice, bob]:
for post in await get_all_posts(user.username):
await like_post(user.username, post.id)

# Get all posts
print("All posts:")
for p in await get_all_posts():
print("-", p.content, f"({p.likes} likes)")
# List users before mutation
print("\nUsers before mutation:")
for user in await users.find().all():
print(f"- {user.username}: {user.followers} followers, pinned_post likes: {user.pinned_post.likes}")

# Run mutation-based update
await reward_active_users(bonus=5)

# List users after mutation
print("\nUsers after mutation:")
for user in await users.find().all():
print(f"- {user.username}: {user.followers} followers, pinned_post likes: {user.pinned_post.likes}")
print(f" Posts: {[p.content for p in user.posts]}")

await driver.close()


async def get_all_posts(username: Optional[str] = None) -> List[Post]:
if username:
user = await users.afind_one(users.username == username)
return user.posts if user else []

all_posts: list[Post] = []
for user in await users.find().all():
all_posts.extend(user.posts)
return all_posts


if __name__ == "__main__":
asyncio.run(main())
Loading
Loading