Skip to content

Adds ability to run parallel windows with multiprocessing - #39

Closed
BigweldIndustries wants to merge 2 commits into
SeanDaBlack:mainfrom
BigweldIndustries:Multiproccessing
Closed

Adds ability to run parallel windows with multiprocessing#39
BigweldIndustries wants to merge 2 commits into
SeanDaBlack:mainfrom
BigweldIndustries:Multiproccessing

Conversation

@BigweldIndustries

Copy link
Copy Markdown
Contributor

This is a big change that can greatly increase speed. I used the futures library to bring multiprocessing to the application. When it starts, it asks how many parallel processes you would like. It opens that amount of windows, each one generates just as it normally would. I made one slight change, that being start_driver() is now initialize_driver() due to the fact they each seperate process needs to start their own driver that does not close each time, and stays open. Same functionality. Also, this is essentially just a re-do of #21, which is very outdated and convoluted. Also this is a more ideal approach, and somewhat like what #16 strived to accomplish. Hopefully in the future we can achieve headless chrome + proxies. I may work on those. Solidarity forever comrades!

@pws1453

pws1453 commented Dec 12, 2021

Copy link
Copy Markdown
Contributor

I've built out the headless code fairly well in my PR here #25. I use argparse to give a user the option to pass a --debug option to run headed Chrome, and to otherwise run headless. If you'd like to implement the same strategies I'm using to do this (or change them a bit), you're more than welcome to. Let me know if you do implement this though, so I can close my PR.

@BigweldIndustries

Copy link
Copy Markdown
Contributor Author

I am curious about how exactly you did that, because when I tried making it headless, it was unable to locate certain elements such as buttons.

@pws1453

pws1453 commented Dec 12, 2021

Copy link
Copy Markdown
Contributor

Same issue - Going to return my PR to draft and work it out

@BigweldIndustries

Copy link
Copy Markdown
Contributor Author

I would like to add, another advantage of this implementation is that you can let it run in the background without interfering with you tasks, (the windows are reused and dont auto focus)

@BigweldIndustries

Copy link
Copy Markdown
Contributor Author

Read #54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants