Speed up get_packages_for_component_id and prepare for future improvements#2421
Open
leolost2605 wants to merge 2 commits intomainfrom
Open
Speed up get_packages_for_component_id and prepare for future improvements#2421leolost2605 wants to merge 2 commits intomainfrom
leolost2605 wants to merge 2 commits intomainfrom
Conversation
acdd25f to
667de74
Compare
667de74 to
206c13d
Compare
206c13d to
5b38dcd
Compare
24c0d89 to
d098547
Compare
Member
Author
|
Ok should be ready again. I wanted to improve how we add components to the component store a bit and then added some comments because we have to be a bit careful about what we can do on a worker thread and what not and in what order we do things. |
Instead of just having a list with all packages introduce a list of a new Component class. Components are just a container that groups all packages for the same appstream component (e.g. from different flatpak remotes). Then we flatten these packages from components to our package model which we can use as usual. This has the major advantage that getting all packages for a given component (which we do quite often) is now a lookup in constant time instead of O(n) in all packages. It also gives us fast access to a listmodel of unique packages which can be used for category, author, search, etc. views.
From O(n) to O(1) since we now have a hashmap with the component ids.
d098547 to
b1eaae5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ok this is a wall of text for really not that many lines of code that were changed but it goes a bit in depth about improvements and some future plans I have :)
Includes #2419
Introduce a component class that holds all packages that belong to a component. Then introduce a hashmap that maps component ids to the new component class.
Because the component class also implements listmodel we can flatten it to the packages instead of having a separate package store. This has the advantage that we can in the future much more easily obtain a package list with only unique packages by just mapping the components to the package with the highest origin score instead of flattening it to all packages. This can be done only a single time in the backend and allow all calls to profit from this. In the future this will speed up and allow to use listmodels for get_packages_for_author, get_packages_for_category, creating the search, etc.
Right now we also get a major performance gain through the second commit (Speedup get_packages_for_component_id).
This call goes from O(n) to O(1) with n being the number of all packages (>10000 with flathub).
The call happens quite often because we need it to determine the origin packages for a given package which happens e.g. everytime a
InstalledPackageRowis constructed (i.e. at startup for every installed package) and every time an appinfoview is opened.This means that with this PR the startup time is a bit faster (which is ofc not that impactful because most of the time is used for loading the packages but apart from that it reduces time by I would say 70-80% see the attached flamegraphs). But it also improves the time needed to construct an appinfoview which makes browsing a lot smoother IMO because we now always get the smooth animation and it doesn't lag because we take too long to get the packages (at least it felt smoother to me 🤷).
A bit background: I found this through some rough profiling:
On main, zoomed to application.activate :

With this PR once again zoomed to application.activate (it isn't big enough to be really visible):
