A character-level, decoder-only Transformer trained in PyTorch on the first Harry Potter book. The model learns to generate text character by character, producing samples that mimic the style and structure of the magical world of Harry Potter.
This project was built to gain hands-on experience with Transformer internals, attention mechanisms, and training dynamics by implementing core components from scratch rather than relying on high-level abstractions.
Pretrained models are provided in the saved_models/ directory, and text can be generated using the included notebook sample_from_model.ipynb.
Install the necessary dependencies using the requirements file:
pip install -r requirements.txtmodel.py: Contains classes SingleHeadAttention, TransformerBlock, and Transformer built from scratch.load_hp_data.py: Includes a function load_data that loads the text data and makes some augmentations.sample_from_model.ipynb: A notebook that allows sampling from a trained model.saved_models/: Directory containing pretrained models.
"Midnight," said Harry.
"Vernon. Next mornin', Harry, his ough, remind I've got to crate away," said Hagrid, "but what's why my dead, I don' know why want to do, I'm too still a few more asleep. I says that with usually - before I think it'll - a can't these people like his great on, we'll go back to the mail and the hat said, like someone - this better team them."
Hagrid squeaked his eyes at back, too the look at the sister.
"You don't letter," he said. "Good to - listen yer pract down to find out of some who was guarding the Snitch!" said Professor McGonagall. "Don't have you shout as if you're I have it told me you!"
Said Harry.
"Wishing we go back, Deaten, Potter?"
"Seeker, you haven't one, just clearly us a new."
The look as if it he could go spir of the train trouble around again.
"Got to Fadam Pomfus Dumbledore to remember him."
"Hagrid caught nice it."
"He's gone," said Ron. "Don't magic hands slowly. That's yer name owl to see."
Harry standed grabbed at Ron.
"HAng - he's quite sure back you, and you don't tell me. We've player a report of everyone. The scar on years with him.
"You were?" he wasn't like a minute of them, we could get rid and relared their best of Privet Drive had no poken him.
They stepped right of dragons in the Great Hall ripped that once her didn't think about the right curflic. He'dn't spent on the Leg-Locker's Certaurs had had dark body annished kicked out she even was wayed, he wasn't looking for.
Harry seat the table to watch the thought back around him fat very since. There was a name and thim. I suppose it was still have from skitches were or hitting slops. He felt looking centaur. It was not one You-Know-Who are over his moment.
At last, Neville was lettering to the front door in miserable to buy the all of the troll, measure works, and he couldn't hard any right.
"And you learn out," Harry said. "It was first some stupid of an everything into a mosty scar if Owl The Twere of a cart of tea Bludger, I could come back from your notic