diff --git a/CLAUDE.md b/CLAUDE.md index adb2b91..6390bf1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -125,6 +125,26 @@ visit on every launch, roughly doubled each time, and after nine launches was pa the 2 MB Chromium will load — a black screen with no line anywhere, on a view that was fine. Needs only the .NET 6 SDK under `~/.dotnet-local`. +### The site-rules harness + +Any change to `SiteRules` — the per-site images/identity rules, and the "which +site did I come from" walk behind the switch key — is exercised off-device first: + +```sh +tools/siterules/run.sh +``` + +The whole risk in that file is one word: what counts as **one site**. It is in +`src/common`, so it is in all six packages, and both ways of getting it wrong are +silent from a TV — too narrow and a rule stops applying the moment a site hands +you to another of its own hosts, too wide and a suffix test lets +`notinstagram.com` wear `instagram.com`'s settings. The harness holds the shipping +file to the name a site is remembered under, what a rule covers and what it must +not, the third state (a field nobody set follows the browser-wide switch rather +than meaning "off"), a round trip through the disk, a hand-edited file that cannot +stop the browser starting, and the alternation the switch key depends on. Needs +only the .NET 6 SDK under `~/.dotnet-local`. + ### The ad-block harness Any change to `AdHosts`, to `src/nui/adhosts.txt` — the host list the NUI diff --git a/README.md b/README.md index 0cbd92e..8bc6473 100644 --- a/README.md +++ b/README.md @@ -74,7 +74,8 @@ browser simply won't start. Newer sets don't have this restriction. | **Back** | Close overlay → back a page → exit | | **CH ▲ / ▼** | Page up / page down | | **0** | Address bar | -| **1** | Switch how the browser identifies itself, and reload | +| **A / red** | Switch to the site you were just on | +| **1** | Switch how the browser identifies itself, and reload — remembered for this site | | **2** | Pointer style | | **3** | Diagnostics (what the page thinks it's talking to) | | **4** | Give the keys to the page, for its own navigation | @@ -83,7 +84,25 @@ browser simply won't start. Newer sets don't have this restriction. | **7** | Show/hide the key hints | | **8** | Keep this page in favourites (press again to remove) | | **9** | Back to the start screen | -| **Info** | Images on/off — the biggest speed-up on an old set | +| **Info** | Images on/off — the biggest speed-up on an old set — remembered for this site | + +Not every remote sends the colour buttons, and the slim ones have none. **Switch +site** is the second row of the menu, so hold OK and it is one press away. + +### Each site keeps its own settings + +Images and identity are remembered **for the site you set them on**. Turn images +off on Instagram and identify it as a desktop; Spotify keeps its images and stays +whatever you set it to. Nothing to configure — press the key on the site you mean +it for, and the message on screen names the site it was saved for. + +Press the same keys on the **start screen** — which is on no site — and you are +setting them for every site that has no answer of its own. *Forget this site's +settings* in the menu takes a site back to those. + +A rule saved on `www.instagram.com` covers `instagram.com` and anything under it. +`open.spotify.com` and `accounts.spotify.com` are two different sites, so a +setting made on one doesn't reach the other. ### No number keys on your remote? diff --git a/docs/INTERNALS.md b/docs/INTERNALS.md index 96e6933..55ec40b 100644 --- a/docs/INTERNALS.md +++ b/docs/INTERNALS.md @@ -1514,6 +1514,123 @@ typed. `Store.Init` therefore has to run **before** the keyboard is constructed both builds used to initialise the store afterwards, silently discarding the user's layout choice. +## Settings that belong to a site, not to the browser + +Issues #74 and #75, from the same reporter, four days apart. Both are the same +question wearing different clothes: **what counts as one site?** + +#74: he wants images off on Instagram and on for Spotify, which will not play +without them, and he wants Instagram identified as a desktop because its mobile +layout is unusable on a television while other sites are better off as mobile. +Two switches, opposite answers, and one switch each — so he was flipping both by +hand on every crossing between the two sites he actually uses. A setting that has +to be redone every time is one the user is maintaining on our behalf. + +#75: "is there any way to switch between two sites fast without typing its url or +going to the start page?" — the same two sites, and the same crossing. + +`src/common/SiteRules.cs` answers both, so all six packages have it. + +### What a site is called + +`SiteRules.KeyFor` is the whole design, and everything else is bookkeeping around +it. A site is its **host**, lower-cased, without the port, the credentials or a +leading `www.`, and a rule saved under that name answers for that host **and +anything under it**: one set on `www.instagram.com` is stored as `instagram.com` +and also covers `i.instagram.com`. The match is on whole labels, so +`notinstagram.com` and `instagram.com.evil.test` are different sites and get +nothing — a plain suffix test would look identical right up until it handed a +stranger's site the settings meant for this one. Where two rules cover the same +host the more specific one wins. + +It is deliberately **not** the registrable domain. There is no public suffix list +in this app and no way to get one onto a 2018 television, and a rule of thumb like +"the last two labels" turns `www.bbc.co.uk` into `co.uk`, which is not a site, it +is a country. So `open.spotify.com` is remembered as itself and +`accounts.spotify.com` is a different site; if that ever needs to be one site, the +answer is a suffix list, not a guess. The screen never leaves this ambiguous — +every message names the site it just saved something for. + +**`KeyFor` returns null for an address that is on no site**, and that half matters +as much as the other. This app's own start screen is one such address, in both of +its shapes (`Store.IsGenerated` knows the marker and the `data:` URL; issue #53 is +what it cost the last time only one of them was recognised), as are `about:` and +`file:`. Null means *leave whatever is applied alone*, so a trip through the start +screen does not reset the settings of the page on either side of it. + +### Unset is not "off" + +A rule holds one field per switch and each may be unset, which is a third state +and not a synonym for the default. A site remembered only for its identity still +follows the browser-wide images switch when that is flipped later. Freezing every +other setting at the moment somebody pressed one key is how a preferences system +starts lying to its user. + +The browser-wide values are still `images` and `uaPreset` in `settings.tsv`, and +they are still what an unremembered site gets. Which one a key press moves is +decided by where it was pressed: **on a page it writes the site's rule, on the +start screen — which is on no site — it moves the browser-wide setting.** That is +the whole of the rule, it needs no extra key, and the flash says which happened +("Images off for instagram.com" against "Images off — faster"). A menu row +forgets a site's rule outright, because otherwise the only way to undo one is to +flip it back to whatever the default happens to be, which leaves the site on the +report as remembered and is not the same thing at all. + +### Applying them costs exactly one reload, and cannot cost two + +Rules go on the view at the start of every load whose site differs from the one +already applied (`ApplySiteRules`), and on our own navigations they go on *before* +the request, so anything this browser opens itself is asked with the right +identity the first time. + +A link followed into another site cannot have that: the engine only reports the +destination once the document has been asked for, and a site serves its mobile or +desktop layout off that request. So a crossing into a site whose identity differs +reloads once — which is exactly the reload the user was doing by hand, once per +crossing, before this existed. The images switch needs no reload of its own; it +takes effect from the point it is set, which at the start of a load is nearly all +of the page. + +The one thing that must not happen is the reload that never stops. Two sites whose +rules disagree and which redirect to each other would reload one another for ever, +and there is no version of that a reporter could describe as anything but "the +browser is broken". So a rule-driven reload is counted, the count resets on any +load that settles without one, and after three the settings are applied without a +reload and the log says so. A page carrying the wrong identity is a bad outcome; a +television that will not stop reloading is a different order of bad. + +### Switching sites + +`SiteRules.OtherSite` walks the history — which is kept most-recent-first — and +returns the first entry that is not on this site. From a page on B that is the +page on A the user came from, and from A it is the page on B they just left, so +one action alternates between the two sites somebody is going back and forth +between. That is #75 as it was asked, and it needs no new screen: a list of recent +sites is what the start screen already is, and the reporter had said he did not +want to go there. + +**It is a menu row before it is a key.** Every digit and `Info` were already spoken +for, so the only button left on a Samsung remote is the red / A one — and which +name that arrives under is undocumented, differs by remote generation, and cannot +be verified here at all, since the slim remotes this app already bends over +backwards for have no colour buttons. `RemoteKeys.SwitchKeys` therefore answers to +every plausible spelling, exactly as `MenuKeys` does, and "Switch site" sits second +in the menu, which every remote can reach. If we guessed the name wrong the button +prints itself on the remote card like any other unrecognised key, which is how we +would find out. + +### What the report says + +`site rules:` names the rule in force on the page in hand, or says nothing is; the +full report at `:8081` lists every remembered site. Both are there because a +setting that follows you around invisibly is the kind that gets reported as "the +browser changed on its own". + +`tools/siterules/run.sh` compiles the shipping file and holds it to all of the +above off-device — the names, what a rule covers and what it must not, the unset +field, the round trip through the disk, a hand-edited file that cannot stop the +browser starting, and the alternation. + ## What the NUI build never asked the engine for The two builds share `src/common` and nothing else, and everything the ElmSharp @@ -2158,11 +2275,13 @@ Issue #20's reporter is on a Tizen 10 set running the NUI package, and is the on person testing that half of this app in anger. **#20 and #53 are closed** as of 2026-09-04: the captcha #20 is named for works (his words, 30 August), the black screen of #53 is fixed and confirmed from his report. This section stays the -state of the 2025 sets across the issues that followed — #37's ad blocking is -the one open thread (his `build-6b29b8e` report is in, `build-f295172`, the -request-trail build that answers it, is the one waiting now), and anything he sends about -`build-df14af8`'s address bar or `build-e9ef92f`'s remote card lands here too. Four builds went out on 2026-09-04 alone; the tag each bullet names is the -one its report has to come from. The state is: +state of the 2025 sets across the issues that followed. **#37's ad blocking is +closed too** — "Ad-block works now", 2026-09-04 — and the open threads are now +#74 and #75, the two things he asked for once the browser stopped breaking: +settings that stay with a site, and a fast way between two of them. Anything he +sends about `build-df14af8`'s address bar or `build-e9ef92f`'s remote card lands +here too. Four builds went out on 2026-09-04 alone; the tag each bullet names is +the one its report has to come from. The state is: - **The session not surviving a restart — fixed.** Shipped in `build-9d856d1`. Nothing pending. @@ -2302,9 +2421,26 @@ one its report has to come from. The state is: (2026-09-04) answered both**, and the second answer stopped mattering when the first one came in. The fix built on it is `AdSilence`: the ad's own host, answered with a second of decodable silence instead of a refusal. Shipped in - `build-2f3862d`. **Waiting on:** his report from that build, taken after an ad - would have played. What each answer decides is under *The ad is on its own host - after all*. + `build-2f3862d`, and **confirmed from the set**: "Ad-block works now" + (2026-09-04). #37 is closed. What the silence has to be, and why it is served + rather than refused, is under *The ad is on its own host after all*. + +- **Settings that stay with a site, and a fast way between two — #74 and #75, + built together.** He wants images off on Instagram and on for Spotify (which + will not play without them), Instagram identified as a desktop and other sites + as mobile, and a way between the two that is not the address bar or the start + screen. `SiteRules` remembers the images and identity switches for the site + they were pressed on and applies them at each crossing; the same notion of "one + site" gives the switch action the site you were just on. Both are in + `src/common`, so all six packages have them, and the ewk builds are the ones + most likely to notice a mistake in them. See *Settings that belong to a site, + not to the browser* above for what a site is called and why the reload is + counted. **Waiting on:** his report from the build that ships them. The two + things worth reading off it are whether a crossing between his two sites now + arrives with the right layout without him pressing anything, and whether the + red / A button on his remote reaches *Switch site* at all — if it does not, the + remote card's footer will be carrying the name his remote actually sends, and + that name is the fix. Five things about that set are settled and should not be re-derived: **key `5` is his, not ours** — the engine's overlay path is the only one that gives him a diff --git a/src/common/RemoteKeys.cs b/src/common/RemoteKeys.cs index 77dd0cb..b6224c9 100644 --- a/src/common/RemoteKeys.cs +++ b/src/common/RemoteKeys.cs @@ -73,6 +73,26 @@ internal static class RemoteKeys PlayBack, AudioPlay, AudioPause, AudioPlayPause, }; + // ---------------------------------------------------------- switch site + // + // The red / A button, which is the only button left on a Samsung remote + // that this app has no use for — every digit and Info are spoken for, and + // issue #75 asked for a way between two sites that is faster than typing + // an address or going through the start screen. + // + // Which name it arrives under is undocumented and differs by remote + // generation (the TV web runtime calls it ColorF0Red; the X keysym is + // XF86Red), so this answers to every plausible spelling, exactly as + // MenuKeys does. None of them is verified on hardware — a slim remote has + // no colour buttons at all — so this is an accelerator and never the only + // way in: "Switch site" sits second in the menu, which every remote can + // reach. A name we guessed wrong prints itself on the remote card like any + // other unknown button, which is how we would find out. + public static readonly string[] SwitchKeys = + { + "XF86Red", "XF86ColorF0Red", "ColorF0Red", + }; + public const string ChannelUp = "XF86RaiseChannel"; public const string ChannelDown = "XF86LowerChannel"; @@ -98,6 +118,7 @@ internal static class RemoteKeys Num0, Num1, Num2, Num3, Num4, Num5, Num6, Num7, Num8, Num9, Tools, SimpleMenu, SysMenu, PlayBack, AudioPlay, AudioPause, AudioPlayPause, + "XF86Red", "XF86ColorF0Red", "ColorF0Red", }; /// @@ -116,6 +137,12 @@ public static bool IsMediaKey(string key) return Contains(MediaKeys, key); } + /// True for the red / A button — see . + public static bool IsSwitchKey(string key) + { + return Contains(SwitchKeys, key); + } + /// /// True for a button whose press is a reason to show the address bar: /// going back, opening the menu, or one of the numbered shortcuts, each of @@ -146,7 +173,8 @@ public static bool IsChromeKey(string key) return true; } - return Contains(MenuKeys, key) || Contains(MediaKeys, key); + return Contains(MenuKeys, key) || Contains(MediaKeys, key) || + Contains(SwitchKeys, key); } private static bool Contains(string[] keys, string key) diff --git a/src/common/RemoteMenu.cs b/src/common/RemoteMenu.cs index 1483067..68a2172 100644 --- a/src/common/RemoteMenu.cs +++ b/src/common/RemoteMenu.cs @@ -44,6 +44,8 @@ public Item(string id, string label, string shortcut) public const string ActionHome = "home"; public const string ActionBookmark = "bookmark"; public const string ActionIdentity = "identity"; + public const string ActionSwitchSite = "switchsite"; + public const string ActionForgetSite = "forgetsite"; public const string ActionTypeInField = "typefield"; public const string ActionKeysToPage = "keystopage"; public const string ActionFitPage = "fitpage"; diff --git a/src/common/SiteRules.cs b/src/common/SiteRules.cs new file mode 100644 index 0000000..dd7caa6 --- /dev/null +++ b/src/common/SiteRules.cs @@ -0,0 +1,485 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.IO; + +namespace Overscan +{ + /// + /// What one site has been asked to be remembered as: the answers this browser + /// gives that site rather than the ones it gives everywhere. + /// + /// + /// Every field can be , and that is not the same as "off". + /// A rule that says nothing about images leaves the browser-wide setting alone, + /// so a site remembered only for how it is identified still follows the images + /// switch when it is flipped later. + /// + internal sealed class SiteRule + { + public const int Unset = -1; + + public SiteRule(string site) + { + Site = site; + Ua = Unset; + Images = Unset; + } + + /// + /// The host this rule answers for, lower-cased and without a leading + /// www. — see . + /// + public string Site { get; private set; } + + /// Index into the UA presets, or . + public int Ua { get; set; } + + /// 1 for on, 0 for off, or . + public int Images { get; set; } + + public bool IsEmpty + { + get { return Ua == Unset && Images == Unset; } + } + } + + /// + /// Per-site settings: which identity a site is given and whether it gets + /// images, remembered for that site alone. + /// + /// Issue #74. The two switches this browser has that change what a site sends + /// back are exactly the two that want a different answer per site: Instagram's + /// mobile layout is unusable on a TV so it wants the desktop identity, while + /// Spotify will not play at all without images, and the reporter was flipping + /// both by hand on every switch between them. A setting that has to be redone + /// every time is one the user is maintaining for us. + /// + /// Stored in sites.tsv, one line per site, in the same shape as + /// everything else keeps and for the same reason: the + /// tizen50 build has no JSON in its framework. + /// + /// open.spotify.com <TAB> images=1 + /// instagram.com <TAB> ua=1 <TAB> images=0 + /// + /// Failures are swallowed and logged. A browser that will not start because a + /// preference file is unreadable is a worse browser than one that forgets a + /// preference. + /// + internal static class SiteRules + { + /// + /// A ceiling on the file, not on the feature. Rules are only ever written + /// by a person pressing a key, so this is unreachable in ordinary use; it + /// is here so that a fault which somehow writes one per page cannot grow a + /// file the next launch has to read. + /// + private const int RuleLimit = 200; + + private const string FileName = "sites.tsv"; + + private static string _dir; + private static readonly List Rules = new List(); + + public static void Init(string dataDirectory) + { + _dir = dataDirectory; + Load(); + DiagLog.Add("site rules: " + Rules.Count + " site(s) remembered"); + } + + public static IList All + { + get { return Rules; } + } + + /// + /// The name a site is remembered under: its host, lower-cased, without the + /// port and without a leading www., or null when the address is not + /// a site at all. + /// + /// Null is the important half. The start screen is a page this app + /// generates rather than somewhere the user went ( + /// knows both shapes of it), and neither it nor an about: page has a + /// site whose settings could be meant. Callers treat null as "leave whatever + /// is applied alone", which is what keeps a trip through the start screen + /// from resetting the settings of the page on either side of it. + /// + public static string KeyFor(string url) + { + if (string.IsNullOrEmpty(url) || url == "-" || Store.IsGenerated(url)) + { + return null; + } + + int scheme = url.IndexOf("://", StringComparison.Ordinal); + if (scheme <= 0) + { + return null; + } + + string protocol = url.Substring(0, scheme).ToLowerInvariant(); + if (protocol != "http" && protocol != "https") + { + return null; + } + + string rest = url.Substring(scheme + 3); + int end = rest.Length; + for (int i = 0; i < rest.Length; i++) + { + char c = rest[i]; + if (c == '/' || c == '?' || c == '#') + { + end = i; + break; + } + } + + string host = rest.Substring(0, end); + + // Credentials, then the port. Both are part of the authority and + // neither is part of the site's name. + int at = host.LastIndexOf('@'); + if (at >= 0) + { + host = host.Substring(at + 1); + } + + int colon = host.IndexOf(':'); + if (colon >= 0) + { + host = host.Substring(0, colon); + } + + host = host.ToLowerInvariant().TrimEnd('.'); + if (host.StartsWith("www.", StringComparison.Ordinal) && host.Length > 4) + { + host = host.Substring(4); + } + + return host.Length == 0 ? null : host; + } + + /// + /// The rule that answers for this address, or null. + /// + /// A rule matches its own host and anything under it, so one saved on + /// instagram.com (which is where www.instagram.com is + /// remembered) also answers for i.instagram.com. The match is on + /// whole labels — notinstagram.com is a different site and a + /// suffix test that let it through would hand a stranger's site the + /// settings meant for this one. The most specific rule wins, so a site + /// with a rule of its own is not overruled by one on its parent. + /// + public static SiteRule For(string url) + { + string host = KeyFor(url); + if (host == null) + { + return null; + } + + SiteRule best = null; + for (int i = 0; i < Rules.Count; i++) + { + string site = Rules[i].Site; + bool covers = host.Length == site.Length + ? string.Equals(host, site, StringComparison.Ordinal) + : host.Length > site.Length && + host[host.Length - site.Length - 1] == '.' && + host.EndsWith(site, StringComparison.Ordinal); + + if (covers && (best == null || site.Length > best.Site.Length)) + { + best = Rules[i]; + } + } + + return best; + } + + /// + /// What this address should be loaded with: the site's own answers where it + /// has them, the browser-wide settings everywhere else. + /// + public static void Effective(string url, int defaultUa, bool defaultImages, + out int ua, out bool images) + { + ua = defaultUa; + images = defaultImages; + + SiteRule rule = For(url); + if (rule == null) + { + return; + } + + if (rule.Ua != SiteRule.Unset) + { + ua = rule.Ua; + } + + if (rule.Images != SiteRule.Unset) + { + images = rule.Images == 1; + } + } + + /// Remembers a UA preset for this site. Returns the rule it landed on. + public static SiteRule SetUa(string url, int preset) + { + SiteRule rule = Claim(url); + if (rule != null) + { + rule.Ua = preset; + Save(); + } + + return rule; + } + + /// Remembers the images switch for this site. Returns the rule it landed on. + public static SiteRule SetImages(string url, bool on) + { + SiteRule rule = Claim(url); + if (rule != null) + { + rule.Images = on ? 1 : 0; + Save(); + } + + return rule; + } + + /// + /// Drops whatever was remembered for this site. Returns the rule that was + /// removed, or null when the site had none — which is what lets the caller + /// say so rather than claiming to have forgotten something. + /// + public static SiteRule Forget(string url) + { + SiteRule rule = For(url); + if (rule == null) + { + return null; + } + + Rules.Remove(rule); + Save(); + return rule; + } + + /// + /// The most recent entry in the history that is on a different site from + /// the address given, or null when there is none. + /// + /// Issue #75, and the reason it is this and not a list: the history is kept + /// most-recent-first, so from a page on B the first entry that is not on B + /// is the page on A the user came from — and once they are on A the first + /// entry that is not on A is the page on B they just left. One action + /// therefore alternates between the two sites somebody is going back and + /// forth between, which is the question as it was asked. From the start + /// screen (a null address, since it is on no site) it opens the most recent + /// site instead. + /// + public static Bookmark OtherSite(IList history, string currentUrl) + { + if (history == null) + { + return null; + } + + string here = KeyFor(currentUrl); + for (int i = 0; i < history.Count; i++) + { + string there = KeyFor(history[i].Url); + if (there == null) + { + continue; + } + + if (here == null || !string.Equals(there, here, StringComparison.Ordinal)) + { + return history[i]; + } + } + + return null; + } + + /// The rules as the diagnostics report shows them. + public static string Dump() + { + if (Rules.Count == 0) + { + return "(none — images and identity are remembered per site once you set them)"; + } + + var text = new System.Text.StringBuilder(); + for (int i = 0; i < Rules.Count; i++) + { + SiteRule rule = Rules[i]; + text.Append(rule.Site); + if (rule.Ua != SiteRule.Unset) + { + text.Append(" identity ").Append(rule.Ua.ToString(CultureInfo.InvariantCulture)); + } + + if (rule.Images != SiteRule.Unset) + { + text.Append(" images ").Append(rule.Images == 1 ? "on" : "off"); + } + + text.Append("\n"); + } + + return text.ToString(); + } + + /// + /// The rule to write this address's answer into: the one already covering + /// it if there is one, a new one for its own host otherwise. + /// + /// Editing the covering rule rather than making a narrower one is what + /// keeps the file honest about what the user did. Somebody who set an + /// identity on instagram.com and later flips images while sitting on + /// i.instagram.com means the same site, and two rules — one of them + /// silently overruling the other — is not something the screen could ever + /// explain to them. + /// + private static SiteRule Claim(string url) + { + SiteRule existing = For(url); + if (existing != null) + { + return existing; + } + + string host = KeyFor(url); + if (host == null || Rules.Count >= RuleLimit) + { + return null; + } + + var rule = new SiteRule(host); + Rules.Add(rule); + return rule; + } + + private static void Load() + { + Rules.Clear(); + if (_dir == null) + { + return; + } + + try + { + string path = Path.Combine(_dir, FileName); + if (!File.Exists(path)) + { + return; + } + + foreach (string line in File.ReadAllLines(path)) + { + string[] parts = line.Split('\t'); + if (parts.Length < 1 || parts[0].Length == 0) + { + continue; + } + + var rule = new SiteRule(parts[0].ToLowerInvariant()); + for (int i = 1; i < parts.Length; i++) + { + int split = parts[i].IndexOf('='); + if (split <= 0) + { + continue; + } + + string name = parts[i].Substring(0, split); + int value; + if (!int.TryParse(parts[i].Substring(split + 1), NumberStyles.Integer, + CultureInfo.InvariantCulture, out value)) + { + continue; + } + + if (name == "ua") + { + rule.Ua = value; + } + else if (name == "images") + { + // Anything that is not a 0 or a 1 is a file we did not + // write; leave the field unset rather than inventing an + // answer for it. + rule.Images = value == 0 || value == 1 ? value : SiteRule.Unset; + } + } + + // A line that says nothing is a rule that does nothing, and + // keeping it would show the site on the report as remembered. + if (!rule.IsEmpty) + { + Rules.Add(rule); + } + } + } + catch (Exception ex) + { + DiagLog.Add("site rules: cannot read " + FileName + ": " + ex.Message); + } + } + + private static void Save() + { + if (_dir == null) + { + return; + } + + // A rule emptied by hand (both fields taken back to the default) is not + // a rule; dropping it here is what stops the file, and the report, from + // filling up with sites that are remembered as "nothing in particular". + for (int i = Rules.Count - 1; i >= 0; i--) + { + if (Rules[i].IsEmpty) + { + Rules.RemoveAt(i); + } + } + + try + { + var lines = new List(); + foreach (SiteRule rule in Rules) + { + var line = new System.Text.StringBuilder(rule.Site); + if (rule.Ua != SiteRule.Unset) + { + line.Append('\t').Append("ua=") + .Append(rule.Ua.ToString(CultureInfo.InvariantCulture)); + } + + if (rule.Images != SiteRule.Unset) + { + line.Append('\t').Append("images=") + .Append(rule.Images.ToString(CultureInfo.InvariantCulture)); + } + + lines.Add(line.ToString()); + } + + File.WriteAllLines(Path.Combine(_dir, FileName), lines.ToArray()); + } + catch (Exception ex) + { + DiagLog.Add("site rules: cannot write " + FileName + ": " + ex.Message); + } + } + } +} diff --git a/src/elm/BrowserApp.cs b/src/elm/BrowserApp.cs index c6ee955..f627fbb 100644 --- a/src/elm/BrowserApp.cs +++ b/src/elm/BrowserApp.cs @@ -62,6 +62,33 @@ internal sealed class BrowserApp : CoreUIApplication private bool _atHome; private bool _hintsWanted = true; private bool _imagesOn = true; + + /// + /// The browser-wide answers, i.e. what a site with no rule of its own + /// gets. and are what + /// the view has actually been told, which on a site with a rule is not the + /// same thing (issue #74). + /// + private bool _defaultImagesOn = true; + private int _defaultPresetIndex; + + /// + /// The site whose rules are on the view, or null while nothing but our own + /// generated pages has been shown. Compared against the site of each load + /// to decide whether anything has to change at all — without it every page + /// of a site would re-apply and re-reload the same settings. + /// + private string _siteKey; + + /// + /// Reloads asked for by a site rule since the last load that settled, and + /// whether this load asked for one. See : two + /// sites whose rules disagree and which redirect to each other would + /// otherwise reload one another for ever. + /// + private int _ruleReloads; + private bool _ruleReloadIssued; + private const int RuleReloadLimit = 3; private IntPtr _tick; private WebView _web; private VirtualCursor _cursor; @@ -130,6 +157,7 @@ protected override void OnCreate() // time KeyboardLayouts is touched, so a Store initialised after it was // built silently threw the user's layout choice away. Store.Init(DirectoryInfo.Data); + SiteRules.Init(DirectoryInfo.Data); Breadcrumbs.Drop("OnCreate: building UI"); BuildUi(); Breadcrumbs.Drop("OnCreate: UI built"); @@ -226,7 +254,8 @@ protected override void OnCreate() _cursor.ToggleVisual(); } - ApplyPreset(Math.Min(Store.GetInt("uaPreset", 0), _presets.Length - 1)); + _defaultPresetIndex = Math.Min(Store.GetInt("uaPreset", 0), _presets.Length - 1); + ApplyPreset(_defaultPresetIndex); // A start page, if one was set with the keyboard's `start` key. Asked // for in issue #15: a browser that always opens on its own start screen @@ -684,7 +713,9 @@ private void BuildHints() int width = 680; // One row taller than the rows need, because the last line of this card // is the unrecognised-button report and it must not fall off the bottom. - int height = 720; + // Grows with the list: a card that gains a key and not a row draws that + // key over the report line. + int height = 760; int left = screen.Width - width - 40; int top = screen.Height - height - 40; @@ -733,7 +764,8 @@ private void DrawHints() new[] { "hold OK", "menu — every action, on any remote" }, new[] { "Ch up/down", "scroll a page" }, new[] { "0", "type an address" }, - new[] { "1", "identify as… — a lighter site, or desktop" }, + new[] { "A / red", "switch to the site you were just on" }, + new[] { "1", "identify as… — kept for this site" }, new[] { "2", "pointer style" }, new[] { "3", "diagnostics — the report at :8081" }, new[] { "4", "keys to page — when a search box is dead" }, @@ -742,7 +774,7 @@ private void DrawHints() new[] { "7", "hide this card" }, new[] { "8", "keep this page as a tile" }, new[] { "9", "start screen" }, - new[] { "Info", "images off — the one real speed-up" }, + new[] { "Info", "images off — kept for this site" }, }; var text = new System.Text.StringBuilder(); @@ -799,6 +831,7 @@ private void BuildMenu() _menu = new RemoteMenu(new[] { new RemoteMenu.Item(RemoteMenu.ActionAddress, "Go to address…", "0"), + new RemoteMenu.Item(RemoteMenu.ActionSwitchSite, "Switch site", "A"), new RemoteMenu.Item(RemoteMenu.ActionHome, "Start screen", "9"), new RemoteMenu.Item(RemoteMenu.ActionBookmark, "Keep this page", "8"), new RemoteMenu.Item(RemoteMenu.ActionTypeInField, "Type in a field…", "5"), @@ -806,6 +839,7 @@ private void BuildMenu() new RemoteMenu.Item(RemoteMenu.ActionKeysToPage, "Send keys to page", "4"), new RemoteMenu.Item(RemoteMenu.ActionFitPage, "Fit page to screen", "6"), new RemoteMenu.Item(RemoteMenu.ActionImages, "Images on/off", "Info"), + new RemoteMenu.Item(RemoteMenu.ActionForgetSite, "Forget this site's settings", string.Empty), new RemoteMenu.Item(RemoteMenu.ActionPointer, "Pointer style", "2"), new RemoteMenu.Item(RemoteMenu.ActionHints, "Remote card on/off", "7"), new RemoteMenu.Item(RemoteMenu.ActionDiagnostics, "Diagnostics", "3"), @@ -966,9 +1000,15 @@ private void RunAction(string id) break; case RemoteMenu.ActionIdentity: - ApplyPreset((_presetIndex + 1) % _presets.Length); - Store.Set("uaPreset", _presetIndex); - _web.Reload(); + CycleIdentity(); + break; + + case RemoteMenu.ActionSwitchSite: + SwitchSite(); + break; + + case RemoteMenu.ActionForgetSite: + ForgetSite(); break; case RemoteMenu.ActionKeysToPage: @@ -1105,7 +1145,7 @@ private void ConfigureWebView() // (Only members present since API 4 are used here, so the same // source compiles for the tizen50 build.) settings.JavaScriptEnabled = true; - _imagesOn = Store.GetBool("images", true); + _imagesOn = _defaultImagesOn = Store.GetBool("images", true); settings.LoadImageAutomatically = _imagesOn; } @@ -1116,6 +1156,11 @@ private void ConfigureWebView() { DiagLog.Add("load started"); _loading = true; + + // Cleared here rather than after the reload is asked for: what the + // limit counts is loads that could not settle, and a load nobody + // had to reload is what settling looks like. + _ruleReloadIssued = false; ShowChrome(); }; _web.LoadFinished += (s, e) => @@ -1128,6 +1173,16 @@ private void ConfigureWebView() ApplyViewportFix(); ReportMetrics(); Store.RecordVisit(_web.Url, _web.Title); + + // Backstop for UrlChanged, which is where a site's rules normally + // land. A page that somehow arrives without one still gets them, + // one load later than it should have. + ApplySiteRules(_web.Url, true); + if (!_ruleReloadIssued) + { + _ruleReloads = 0; + } + UpdateStatus(); }; _web.LoadError += (s, e) => @@ -1139,7 +1194,13 @@ private void ConfigureWebView() }; _web.UrlChanged += (s, e) => { - _urlLabel.Text = Markup(e.GetAsString()); + // The first point at which the destination of a click is known. + // ewk reports no URL on LoadStarted, so this is where a site's own + // settings are put on the view — see ApplySiteRules for why that is + // early enough for the images and one reload late for the identity. + string arrived = e.GetAsString(); + _urlLabel.Text = Markup(arrived); + ApplySiteRules(arrived, true); UpdateStatus(); }; _web.TitleChanged += (s, e) => UpdateStatus(); @@ -1359,7 +1420,15 @@ private void OnKeyDown(object sender, EvasKeyEventArgs e) break; default: - NoteUnknownKey(key); + if (RemoteKeys.IsSwitchKey(key)) + { + RunAction(RemoteMenu.ActionSwitchSite); + } + else + { + NoteUnknownKey(key); + } + break; } } @@ -1429,25 +1498,258 @@ private void ShowHome() /// /// Images off is the single biggest speed-up available on an old set — the /// engine cannot be made faster, but it can be given much less to do. + /// + /// Remembered for the site it was pressed on, not for the browser (issue + /// #74): the reporter turns images off on Instagram and needs them on for + /// Spotify, which will not play without them, and one switch for both means + /// flipping it by hand on every crossing. Pressed on the start screen — + /// which is on no site — it moves the browser-wide setting instead, so + /// there is still a way to say "everywhere". /// private void ToggleImages() { - _imagesOn = !_imagesOn; - Store.Set("images", _imagesOn); + bool on = !_imagesOn; + SiteRule rule = SiteRules.SetImages(PageUrl(), on); + if (rule == null) + { + _defaultImagesOn = on; + Store.Set("images", on); + + // The browser-wide answer moved, so what is on the view is no + // longer what any site was last worked out to want. Forgetting + // which site that was is what makes the next load work it out + // again — otherwise a site with a rule, left through the start + // screen and come back to, keeps the setting made while away. + _siteKey = null; + Flash(on ? "Images on" : "Images off — faster"); + } + else + { + Flash((on ? "Images on for " : "Images off for ") + rule.Site); + } + + ApplyImages(on); + Reload("images"); + } + + /// + /// The next identity in the list, remembered for this site. Same split as + /// the images switch: on a site it is that site's answer, on the start + /// screen it is the browser's. + /// + private void CycleIdentity() + { + int next = (_presetIndex + 1) % _presets.Length; + ApplyPreset(next); + + SiteRule rule = SiteRules.SetUa(PageUrl(), next); + if (rule == null) + { + _defaultPresetIndex = next; + Store.Set("uaPreset", next); + + // See ToggleImages: a moved browser-wide answer invalidates the + // site the view was last set up for. + _siteKey = null; + Flash(ShortPreset(_presets[next].Label)); + } + else + { + Flash(ShortPreset(_presets[next].Label) + " for " + rule.Site); + } + + Reload("identity"); + } + + /// + /// Opens the most recent page that is not on this site — issue #75. From a + /// page on B that is where the user came from, and from there it is B + /// again, so the one action goes back and forth between the two sites + /// somebody is actually using. + /// + private void SwitchSite() + { + Bookmark other = SiteRules.OtherSite(Store.RecentHistory, PageUrl()); + if (other == null) + { + Flash("No other site to switch to yet"); + return; + } + + Flash("\u2192 " + SiteRules.KeyFor(other.Url)); + Navigate(other.Url); + } + + /// + /// Takes this site back to the browser-wide answers. Without it a rule can + /// only be undone by flipping it back to whatever the default happens to + /// be, which is not the same thing and leaves the site on the report as + /// remembered. + /// + private void ForgetSite() + { + string url = PageUrl(); + SiteRule rule = SiteRules.Forget(url); + if (rule == null) + { + Flash(url == null ? "Open a site first" : "Nothing is remembered for this site"); + return; + } + + DiagLog.Add("site rules: forgot " + rule.Site); + Flash("Forgot the settings for " + rule.Site); + + // A change of settings like any other, so it is applied and reloaded + // like any other. The forgotten site is no longer the applied one. + _siteKey = null; + ApplySiteRules(url, true); + } + + /// + /// Puts a site's own answers on the view when the load is for a different + /// site from the one already applied. + /// + /// The images switch takes effect from here on, which at the start of a + /// load is nearly all of the page. The identity cannot: the document was + /// already asked for with whatever was applied before, and a site serves + /// its mobile or desktop layout off that request. So a crossing into a site + /// whose identity differs costs one reload — which is exactly the reload + /// the user was doing by hand, once per crossing, before this existed. + /// + /// is false on our own navigations, where the + /// load has not gone out yet and the settings are simply right from the + /// first request. + /// + private void ApplySiteRules(string url, bool mayReload) + { + // Our own generated pages are on no site (KeyFor returns null for them) + // and leave the settings alone, so a trip through the start screen does + // not reset the site on either side of it. + string site = SiteRules.KeyFor(url); + if (site == null || site == _siteKey) + { + return; + } + + _siteKey = site; + + int ua; + bool images; + SiteRules.Effective(url, _defaultPresetIndex, _defaultImagesOn, out ua, out images); + if (ua < 0 || ua >= _presets.Length) + { + // A rule written by a build with more presets than this one. + ua = _defaultPresetIndex; + } + + bool uaChanged = ua != _presetIndex; + bool imagesChanged = images != _imagesOn; + if (!uaChanged && !imagesChanged) + { + return; + } + + if (uaChanged) + { + ApplyPreset(ua); + } + + if (imagesChanged) + { + ApplyImages(images); + } + + DiagLog.Add("site rules for " + site + ": " + ShortPreset(_presets[ua].Label) + + ", images " + (images ? "on" : "off")); + + if (!mayReload) + { + return; + } + + // Two sites whose rules disagree and which redirect to each other would + // reload one another for ever, and an unusable browser is a far worse + // outcome than a page carrying the wrong identity. The limit is per run + // of loads that never settle: an ordinary crossing reloads once and the + // load after it resets the count. + if (_ruleReloads >= RuleReloadLimit) + { + DiagLog.Add("site rules: not reloading again for " + site + + " — " + _ruleReloads + " reloads without a page settling"); + return; + } + + _ruleReloads++; + _ruleReloadIssued = true; + + // Not from here: this is normally reached from inside ewk's own + // UrlChanged notification, and asking the engine to start a new load + // while it is telling us about the one it just started is the kind of + // re-entry this platform does not forgive. One turn of the main loop + // later is still long before any of the page has been painted. + try + { + EcoreMainloop.Post(delegate { Reload("site rules"); }); + } + catch (Exception ex) + { + // No main-loop post means no deferral. The reload is worth more + // than the caution, so it happens here instead. + DiagLog.Add("deferring the site-rules reload failed: " + ex.Message); + Reload("site rules"); + } + } + + /// Tells the view about the images switch. Best-effort, like everything past it. + private void ApplyImages(bool on) + { + _imagesOn = on; try { Settings settings = _web.GetSettings(); if (settings != null) { - settings.LoadImageAutomatically = _imagesOn; + settings.LoadImageAutomatically = on; } + } + catch (Exception ex) + { + DiagLog.Add("image switch failed: " + ex.Message); + } + } - Flash(_imagesOn ? "Images on" : "Images off — faster"); + private void Reload(string why) + { + try + { _web.Reload(); } catch (Exception ex) { - DiagLog.Add("image toggle failed: " + ex.Message); + DiagLog.Add("reload after " + why + " failed: " + ex.Message); + } + } + + /// + /// The address the page is on, or null when it is one of ours. Everything + /// per-site keys off this, and "no site" is a real answer rather than a + /// missing one — see . + /// + private string PageUrl() + { + if (_web == null || _atHome) + { + return null; + } + + try + { + return _web.Url; + } + catch (Exception ex) + { + DiagLog.Add("reading the page url failed: " + ex.Message); + return _cachedUrl; } } @@ -1613,6 +1915,10 @@ private void Navigate(string input) string url = Urls.Normalize(input); _atHome = false; DiagLog.Add("navigate: " + url); + + // Before the request, so a site we are opening ourselves is asked with + // its own identity the first time and needs no reload at all. + ApplySiteRules(url, false); _web.LoadUrl(url); _cursor.Center(); UpdateStatus(); @@ -1687,7 +1993,8 @@ private void UpdateStatus() ShortPreset(_presets[_presetIndex].Label) + " · " + (_keysToPage ? "page keys" : "cursor") + (_viewportFix ? " · fit" : string.Empty) + - (_imagesOn ? string.Empty : " · no images"); + (_imagesOn ? string.Empty : " · no images") + + (SiteRules.For(_cachedUrl) == null ? string.Empty : " · for this site"); _status.Text = Theme.Text(state, 24, Theme.InkMuted, false, "right"); } @@ -1813,10 +2120,14 @@ private string DiagnosticsText(bool full) "last click: " + _lastClick + "\n" + "frame click: " + NativeMouse.LastResult + "\n" + "start page: " + (string.IsNullOrEmpty(_startupUrl) ? "(start screen)" : _startupUrl) + "\n" + + "site rules: " + (SiteRules.For(_cachedUrl) == null + ? "nothing remembered for this site" + : "in force for " + SiteRules.For(_cachedUrl).Site) + "\n" + "cursor : " + (_cursor == null ? "(not built)" : _cursor.Visual.ToString()) + ", keys -> " + (_keysToPage ? "page" : "cursor") + "\n" + "title : " + _cachedTitle + "\n" + "url : " + _cachedUrl + "\n" + + (full ? "\nsites remembered (issue 74)\n" + SiteRules.Dump() : string.Empty) + trail; } diff --git a/src/nui/NuiBrowserApp.cs b/src/nui/NuiBrowserApp.cs index e456d91..ebda0c7 100644 --- a/src/nui/NuiBrowserApp.cs +++ b/src/nui/NuiBrowserApp.cs @@ -100,6 +100,33 @@ internal sealed class NuiBrowserApp : NUIApplication private bool _hintsWanted = true; private bool _imagesOn = true; private bool _adBlockOn = true; + + /// + /// The browser-wide answers, i.e. what a site with no rule of its own + /// gets. and are what + /// the view has actually been told, which on a site with a rule is not the + /// same thing (issue #74). + /// + private bool _defaultImagesOn = true; + private int _defaultPresetIndex; + + /// + /// The site whose rules are on the view, or null while nothing but our own + /// generated pages has been shown. Compared against the site of each load + /// to decide whether anything has to change at all — without it every page + /// of a site would re-apply and re-reload the same settings. + /// + private string _siteKey; + + /// + /// Reloads asked for by a site rule since the last load that settled, and + /// whether this load asked for one. See : two + /// sites whose rules disagree and which redirect to each other would + /// otherwise reload one another for ever. + /// + private int _ruleReloads; + private bool _ruleReloadIssued; + private const int RuleReloadLimit = 3; private bool _videoOverlay = true; /// @@ -227,6 +254,7 @@ protected override void OnCreate() // first time KeyboardLayouts is touched, so initialising the store // afterwards silently threw the user's layout choice away. Store.Init(DirectoryInfo.Data); + SiteRules.Init(DirectoryInfo.Data); BuildChrome(); BuildOverlay(); @@ -255,7 +283,8 @@ protected override void OnCreate() _startupUrl = Store.Get("startupUrl", null); ShowHints(_hintsWanted); - ApplyPreset(Math.Min(Store.GetInt("uaPreset", 0), _presets.Length - 1)); + _defaultPresetIndex = Math.Min(Store.GetInt("uaPreset", 0), _presets.Length - 1); + ApplyPreset(_defaultPresetIndex); // A start page, if one was set with the keyboard's `start` key (issue // #15): otherwise the same address gets typed on a remote every launch. @@ -396,7 +425,7 @@ private bool TryStartEngine() // fit" behaviour, i.e. the suspected cause of the stretched // rendering on older sets. We want 1:1. _web.Settings.AutoFittingEnabled = false; - _imagesOn = Store.GetBool("images", true); + _imagesOn = _defaultImagesOn = Store.GetBool("images", true); _web.Settings.AutomaticImageLoadingAllowed = _imagesOn; // Nothing this browser does is worth a private window, and the @@ -444,6 +473,17 @@ private bool TryStartEngine() Breadcrumbs.Drop("load started: " + SafeUrl()); _loading = true; + // Cleared here rather than after a reload is asked for: what + // the limit counts is loads that could not settle, and a load + // nobody had to reload is what settling looks like. + _ruleReloadIssued = false; + + // The engine reports the destination here, which is where a + // site's own settings go on the view — see ApplySiteRules for + // why that is early enough for the images and one reload late + // for the identity. + ApplySiteRules(SafeUrl(), true); + // The engine answered, so the view is not the dead kind. _loadAskedAt = DateTime.MinValue; if (_blankRecoveries > 0) @@ -478,6 +518,15 @@ private bool TryStartEngine() Probe(); ApplyViewportFix(); Store.RecordVisit(SafeUrl(), SafeTitle()); + + // Backstop, for a load whose start we somehow missed. Normally + // a no-op: the site is already the applied one by now. + ApplySiteRules(SafeUrl(), true); + if (!_ruleReloadIssued) + { + _ruleReloads = 0; + } + UpdateStatus(); }; @@ -1111,6 +1160,30 @@ private void BuildOverlay() _window.Add(_overlay); } + /// + /// Every key this build answers to, one line each. "hold OK" comes first + /// and is not a number, deliberately: on a slim remote it is the only row + /// here the user can act on, and a list that opens with nine things they + /// cannot do reads as "this app is not for your remote" (issue #27). + /// + private static readonly string[][] RemoteRows = + { + new[] { "hold OK", "menu — every action, on any remote" }, + new[] { "Ch up/down", "scroll a page" }, + new[] { "0", "type an address" }, + new[] { "A / red", "switch to the site you were just on" }, + new[] { "1", "identify as… — kept for this site" }, + new[] { "2", "type in the field you clicked" }, + new[] { "3", "diagnostics — the report at :8081" }, + new[] { "4", "keys to page — when a search box is dead" }, + new[] { "5", "video: TV overlay / in page — if black" }, + new[] { "6", "fit page — when the page is cut off" }, + new[] { "7", "hide this card" }, + new[] { "8", "keep this page as a tile" }, + new[] { "9", "start screen" }, + new[] { "Info", "images off — kept for this site" }, + }; + /// /// The remote card is the surface that teaches the keys (issue #38): every /// key this build answers to, each with the one line that says what it is @@ -1122,7 +1195,12 @@ private void BuildHints() { Size2D screen = _window.WindowSize; int width = 640; - int height = 688; + + // Title, one 42px row per key, then a row's worth of clearance for the + // unrecognised-button line that sits on the bottom edge. Derived rather + // than a constant, because a card that grows by a key and not by a row + // draws the last key on top of that line. + int height = NuiTheme.Pad + 52 + ((RemoteRows.Length + 1) * 42) + NuiTheme.Pad; _hints = new View { @@ -1142,26 +1220,7 @@ private void BuildHints() }; _hints.Add(title); - // "hold OK" comes first and is not a number, deliberately. On a slim - // remote it is the only row on this card the user can act on, and a - // list that opens with nine things they cannot do reads as "this app - // is not for your remote" (issue #27). - string[][] rows = - { - new[] { "hold OK", "menu — every action, on any remote" }, - new[] { "Ch up/down", "scroll a page" }, - new[] { "0", "type an address" }, - new[] { "1", "identify as… — a lighter site, or desktop" }, - new[] { "2", "type in the field you clicked" }, - new[] { "3", "diagnostics — the report at :8081" }, - new[] { "4", "keys to page — when a search box is dead" }, - new[] { "5", "video: TV overlay / in page — if black" }, - new[] { "6", "fit page — when the page is cut off" }, - new[] { "7", "hide this card" }, - new[] { "8", "keep this page as a tile" }, - new[] { "9", "start screen" }, - new[] { "Info", "images off — the one real speed-up" }, - }; + string[][] rows = RemoteRows; for (int i = 0; i < rows.Length; i++) { @@ -1227,6 +1286,7 @@ private void BuildMenu() _menu = new RemoteMenu(new[] { new RemoteMenu.Item(RemoteMenu.ActionAddress, "Go to address…", "0"), + new RemoteMenu.Item(RemoteMenu.ActionSwitchSite, "Switch site", "A"), new RemoteMenu.Item(RemoteMenu.ActionHome, "Start screen", "9"), new RemoteMenu.Item(RemoteMenu.ActionBookmark, "Keep this page", "8"), new RemoteMenu.Item(RemoteMenu.ActionTypeInField, "Type in a field…", "2"), @@ -1235,6 +1295,7 @@ private void BuildMenu() new RemoteMenu.Item(RemoteMenu.ActionFitPage, "Fit page to screen", "6"), new RemoteMenu.Item(RemoteMenu.ActionImages, "Images on/off", "Info"), new RemoteMenu.Item(RemoteMenu.ActionAdBlock, "Ad blocking on/off", string.Empty), + new RemoteMenu.Item(RemoteMenu.ActionForgetSite, "Forget this site's settings", string.Empty), new RemoteMenu.Item(RemoteMenu.ActionVideoPath, "Video: in page / overlay", "5"), new RemoteMenu.Item(RemoteMenu.ActionHints, "Remote card on/off", "7"), new RemoteMenu.Item(RemoteMenu.ActionDiagnostics, "Diagnostics", "3"), @@ -1385,9 +1446,15 @@ private void RunAction(string id) break; case RemoteMenu.ActionIdentity: - ApplyPreset((_presetIndex + 1) % _presets.Length); - Store.Set("uaPreset", _presetIndex); - _web.Reload(); + CycleIdentity(); + break; + + case RemoteMenu.ActionSwitchSite: + SwitchSite(); + break; + + case RemoteMenu.ActionForgetSite: + ForgetSite(); break; case RemoteMenu.ActionKeysToPage: @@ -1636,7 +1703,15 @@ private void OnWindowKey(object sender, Window.KeyEventArgs e) break; default: - NoteUnknownKey(key); + if (RemoteKeys.IsSwitchKey(key)) + { + RunAction(RemoteMenu.ActionSwitchSite); + } + else + { + NoteUnknownKey(key); + } + break; } } @@ -1918,6 +1993,10 @@ private void Navigate(string url) _loadAskedAt = DateTime.UtcNow; _loadAskedFor = url; Breadcrumbs.Drop("navigate: " + url); + + // Before the request, so a site we are opening ourselves is asked with + // its own identity the first time and needs no reload at all. + ApplySiteRules(url, false); _web.LoadUrl(url); _cursor.Center(); UpdateStatus(); @@ -1950,27 +2029,241 @@ private void ToggleAdBlock() /// /// The one real speed-up available: the engine cannot be made faster, but it /// can be given much less to do. + /// + /// Remembered for the site it was pressed on, not for the browser (issue + /// #74): the reporter turns images off on Instagram and needs them on for + /// Spotify, which will not play without them, and one switch for both means + /// flipping it by hand on every crossing. Pressed on the start screen — + /// which is on no site — it moves the browser-wide setting instead, so + /// there is still a way to say "everywhere". /// private void ToggleImages() { - _imagesOn = !_imagesOn; - Store.Set("images", _imagesOn); + bool on = !_imagesOn; + SiteRule rule = SiteRules.SetImages(PageUrl(), on); + if (rule == null) + { + _defaultImagesOn = on; + Store.Set("images", on); + + // The browser-wide answer moved, so what is on the view is no + // longer what any site was last worked out to want. Forgetting + // which site that was is what makes the next load work it out + // again — otherwise a site with a rule, left through the start + // screen and come back to, keeps the setting made while away. + _siteKey = null; + Flash(on ? "Images on" : "Images off — faster"); + } + else + { + Flash((on ? "Images on for " : "Images off for ") + rule.Site); + } + + ApplyImages(on); + Reload("images"); + } + + /// + /// The next identity in the list, remembered for this site. Same split as + /// the images switch: on a site it is that site's answer, on the start + /// screen it is the browser's. + /// + private void CycleIdentity() + { + int next = (_presetIndex + 1) % _presets.Length; + ApplyPreset(next); + + SiteRule rule = SiteRules.SetUa(PageUrl(), next); + if (rule == null) + { + _defaultPresetIndex = next; + Store.Set("uaPreset", next); + + // See ToggleImages: a moved browser-wide answer invalidates the + // site the view was last set up for. + _siteKey = null; + Flash(ShortPreset(_presets[next].Label)); + } + else + { + Flash(ShortPreset(_presets[next].Label) + " for " + rule.Site); + } + + Reload("identity"); + } + + /// + /// Opens the most recent page that is not on this site — issue #75. From a + /// page on B that is where the user came from, and from there it is B + /// again, so the one action goes back and forth between the two sites + /// somebody is actually using. + /// + private void SwitchSite() + { + Bookmark other = SiteRules.OtherSite(Store.RecentHistory, PageUrl()); + if (other == null) + { + Flash("No other site to switch to yet"); + return; + } + + Flash("\u2192 " + SiteRules.KeyFor(other.Url)); + Navigate(other.Url); + } + + /// + /// Takes this site back to the browser-wide answers. Without it a rule can + /// only be undone by flipping it back to whatever the default happens to + /// be, which is not the same thing and leaves the site on the report as + /// remembered. + /// + private void ForgetSite() + { + string url = PageUrl(); + SiteRule rule = SiteRules.Forget(url); + if (rule == null) + { + Flash(url == null ? "Open a site first" : "Nothing is remembered for this site"); + return; + } + + DiagLog.Add("site rules: forgot " + rule.Site); + Flash("Forgot the settings for " + rule.Site); + + // A change of settings like any other, so it is applied and reloaded + // like any other. The forgotten site is no longer the applied one. + _siteKey = null; + ApplySiteRules(url, true); + } + + /// + /// Puts a site's own answers on the view when the load is for a different + /// site from the one already applied. + /// + /// The images switch takes effect from here on, which at the start of a + /// load is nearly all of the page. The identity cannot: the document was + /// already asked for with whatever was applied before, and a site serves + /// its mobile or desktop layout off that request. So a crossing into a site + /// whose identity differs costs one reload — which is exactly the reload + /// the user was doing by hand, once per crossing, before this existed. + /// + /// is false on our own navigations, where the + /// load has not gone out yet and the settings are simply right from the + /// first request. + /// + private void ApplySiteRules(string url, bool mayReload) + { + // Our own generated pages are on no site (KeyFor returns null for them) + // and leave the settings alone, so a trip through the start screen does + // not reset the site on either side of it. + string site = SiteRules.KeyFor(url); + if (site == null || site == _siteKey) + { + return; + } + + _siteKey = site; + + int ua; + bool images; + SiteRules.Effective(url, _defaultPresetIndex, _defaultImagesOn, out ua, out images); + if (ua < 0 || ua >= _presets.Length) + { + // A rule written by a build with more presets than this one. + ua = _defaultPresetIndex; + } + + bool uaChanged = ua != _presetIndex; + bool imagesChanged = images != _imagesOn; + if (!uaChanged && !imagesChanged) + { + return; + } + + if (uaChanged) + { + ApplyPreset(ua); + } + + if (imagesChanged) + { + ApplyImages(images); + } + + DiagLog.Add("site rules for " + site + ": " + ShortPreset(_presets[ua].Label) + + ", images " + (images ? "on" : "off")); + + if (!mayReload) + { + return; + } + + // Two sites whose rules disagree and which redirect to each other would + // reload one another for ever, and an unusable browser is a far worse + // outcome than a page carrying the wrong identity. The limit is per run + // of loads that never settle: an ordinary crossing reloads once and the + // load after it resets the count. + if (_ruleReloads >= RuleReloadLimit) + { + DiagLog.Add("site rules: not reloading again for " + site + + " — " + _ruleReloads + " reloads without a page settling"); + return; + } + + _ruleReloads++; + _ruleReloadIssued = true; + + // Not from here: this is normally reached from inside the engine's own + // PageLoadStarted notification, and asking a web engine to start a new + // load while it is telling us about the one it just started is the kind + // of re-entry this platform does not forgive. A tick of the main loop + // later is still long before any of the page has been painted. If no + // timer can be armed the reload is worth more than the caution. + if (!NuiLater.Once(1, delegate { Reload("site rules"); })) + { + Reload("site rules"); + } + } + + /// Tells the view about the images switch. Best-effort, like everything past it. + private void ApplyImages(bool on) + { + _imagesOn = on; try { if (_web.Settings != null) { - _web.Settings.AutomaticImageLoadingAllowed = _imagesOn; + _web.Settings.AutomaticImageLoadingAllowed = on; } + } + catch (Exception ex) + { + DiagLog.Add("image switch failed: " + ex.Message); + } + } - Flash(_imagesOn ? "Images on" : "Images off — faster"); + private void Reload(string why) + { + try + { _web.Reload(); } catch (Exception ex) { - DiagLog.Add("image toggle failed: " + ex.Message); + DiagLog.Add("reload after " + why + " failed: " + ex.Message); } } + /// + /// The address the page is on, or null when it is one of ours. Everything + /// per-site keys off this, and "no site" is a real answer rather than a + /// missing one — see . + /// + private string PageUrl() + { + return _web == null || _atHome ? null : SafeUrl(); + } + private void ToggleFavourite() { if (_atHome) @@ -2111,7 +2404,8 @@ private void UpdateStatus() " · " + (_keysToPage ? "page keys" : "cursor") + (_viewportFix ? " · fit" : string.Empty) + (_imagesOn ? string.Empty : " · no images") + - (_adBlockOn ? string.Empty : " · ads allowed"); + (_adBlockOn ? string.Empty : " · ads allowed") + + (SiteRules.For(_cachedUrl) == null ? string.Empty : " · for this site"); } private string Report() @@ -2135,6 +2429,9 @@ private string Report() "video rect: " + NuiVideoRect.LastBox + "\n" + "blank view: " + _blankState + "\n" + "start page: " + _startPageState + "\n" + + "site rules: " + (SiteRules.For(_cachedUrl) == null + ? "nothing remembered for this site" + : "in force for " + SiteRules.For(_cachedUrl).Site) + "\n" + "ad block : " + NuiAdBlock.Summary() + "\n" + "memory : " + ProcessMemory.Summary() + ", peak " + _peakMemoryMb + " MB\n" + "last words: " + NuiDeathWatch.LastWord + "\n" + @@ -2155,6 +2452,7 @@ private string Report() // and Range both are. The column stays because it costs nothing // and a firmware that starts sending it would be worth knowing // about; the section itself is what found the ad's host. + "sites remembered (issue 74)\n" + SiteRules.Dump() + "\n" + "requests this run (one line per host and first path segment, most first)\n" + RequestTrail.Dump() + "\n\n" + "previous run (last line is where it died)\n" + Breadcrumbs.Previous + "\n\n" + diff --git a/tools/siterules/Program.cs b/tools/siterules/Program.cs new file mode 100644 index 0000000..173b903 --- /dev/null +++ b/tools/siterules/Program.cs @@ -0,0 +1,241 @@ +using System; +using System.Collections.Generic; +using System.IO; + +namespace Overscan +{ + /// Stands in for the app's diagnostics log; the harness reads it back. + internal static class DiagLog + { + public static readonly List Lines = new List(); + + public static void Add(string line) + { + Lines.Add(line); + } + } + + internal static class Program + { + private static int _failures; + + private static void Check(bool ok, string what) + { + Console.WriteLine((ok ? "ok " : "FAIL ") + what); + if (!ok) + { + _failures++; + } + } + + /// + /// What the NUI engine reports for a page loaded from a string: the page + /// itself, percent-encoded, behind a data: prefix. The start screen arrives + /// at KeyFor in this shape on one build and as the marker URL on the other, + /// and both have to come back as "no site". + /// + private static string AsEngineUrl(string html) + { + return "data:text/html;charset=utf-8," + Uri.EscapeDataString(html); + } + + private static string Key(string url) + { + return SiteRules.KeyFor(url) ?? "(none)"; + } + + private static int Main(string[] args) + { + string root = args[0]; + Directory.CreateDirectory(root); + + // ---------------------------------------------------------- 1. names + // + // What a site is called. Getting this wrong in either direction is a + // whole class of fault: too narrow and a rule stops applying the moment + // a site moves you to another of its own hosts, too wide and one site's + // settings land on somebody else's. + Check(Key("https://www.instagram.com/") == "instagram.com", "www. is not part of the name"); + Check(Key("https://INSTAGRAM.com/x") == "instagram.com", "the name is lower-cased"); + Check(Key("https://open.spotify.com:443/?x=1#y") == "open.spotify.com", + "port, query and fragment are not part of the name"); + Check(Key("https://user:pw@example.com/") == "example.com", + "credentials are not part of the name"); + Check(Key("https://example.com./") == "example.com", "a trailing root dot is dropped"); + Check(Key("http://192.168.1.5:8081/") == "192.168.1.5", "an address is a site like any other"); + + // The addresses that are not sites. Both shapes of this app's own start + // screen are in here: issue #53 is what it cost the last time only one + // of them was recognised. + Check(SiteRules.KeyFor(null) == null, "null is not a site"); + Check(SiteRules.KeyFor("-") == null, "the empty-url placeholder is not a site"); + Check(SiteRules.KeyFor("about:blank") == null, "about: is not a site"); + Check(SiteRules.KeyFor(HomePage.BaseUrl) == null, "the start-screen marker is not a site"); + Check(SiteRules.KeyFor(AsEngineUrl("")) == null, + "a data: page is not a site"); + Check(SiteRules.KeyFor("file:///etc/passwd") == null, "file: is not a site"); + + // ------------------------------------------------- 2. what a rule covers + string dir = Path.Combine(root, "cover"); + Directory.CreateDirectory(dir); + SiteRules.Init(dir); + SiteRules.SetUa("https://www.instagram.com/", 1); + + Check(SiteRules.For("https://www.instagram.com/x") != null, "the site itself matches"); + Check(SiteRules.For("https://instagram.com/") != null, "and without the www."); + Check(SiteRules.For("https://i.instagram.com/api") != null, "a host under it matches"); + Check(SiteRules.For("https://notinstagram.com/") == null, + "a look-alike does NOT match — the boundary is a whole label"); + Check(SiteRules.For("https://instagram.com.evil.test/") == null, + "and neither does a site that merely starts with the name"); + Check(SiteRules.For("https://example.com/") == null, "an unrelated site has no rule"); + + // The most specific rule wins where two of them cover the same host. + // Reachable in this order only: a rule made on the subdomain first, + // and one on the site afterwards, which does not yet cover it. The + // other order edits the covering rule instead — check 3 below. + string nested = Path.Combine(root, "nested"); + Directory.CreateDirectory(nested); + SiteRules.Init(nested); + SiteRules.SetUa("https://i.instagram.com/", 3); + SiteRules.SetUa("https://www.instagram.com/", 1); + Check(SiteRules.All.Count == 2 && + SiteRules.For("https://i.instagram.com/").Ua == 3 && + SiteRules.For("https://www.instagram.com/").Ua == 1, + "the more specific rule wins where both cover a host"); + + // ------------------------------------------------- 3. what a rule says + // + // Unset is not "off". A site remembered only for its identity still + // follows the browser-wide images switch when that is flipped later, + // which is the difference between remembering a choice and freezing + // every other one alongside it. + string prefs = Path.Combine(root, "prefs"); + Directory.CreateDirectory(prefs); + SiteRules.Init(prefs); + SiteRules.SetUa("https://open.spotify.com/", 2); + + int ua; + bool images; + SiteRules.Effective("https://open.spotify.com/", 0, false, out ua, out images); + Check(ua == 2 && !images, "the site's identity is used and the browser's images are kept"); + + SiteRules.Effective("https://open.spotify.com/", 0, true, out ua, out images); + Check(ua == 2 && images, "and the browser's images the other way round too"); + + SiteRules.SetImages("https://open.spotify.com/", true); + SiteRules.Effective("https://open.spotify.com/", 0, false, out ua, out images); + Check(ua == 2 && images, "once set, the site's images win over the browser's"); + + SiteRules.Effective("https://example.com/", 4, false, out ua, out images); + Check(ua == 4 && !images, "a site with no rule gets the browser's answers unchanged"); + + SiteRules.Effective(HomePage.BaseUrl, 4, true, out ua, out images); + Check(ua == 4 && images, "and so does an address that is on no site at all"); + + // A second switch on the same site edits the same rule rather than + // adding another one beside it. + Check(SiteRules.All.Count == 1, "one site set twice is still one rule"); + + // A rule is claimed at the level that already covers the host, so a + // switch flipped on a subdomain does not quietly shadow the rule the + // user made on the site. + SiteRule onSub = SiteRules.SetImages("https://accounts.spotify.com/", false); + Check(onSub != null && onSub.Site == "accounts.spotify.com" && SiteRules.All.Count == 2, + "a host no rule covers gets one of its own"); + SiteRule again = SiteRules.SetUa("https://widget.open.spotify.com/", 3); + Check(again != null && again.Site == "open.spotify.com" && SiteRules.All.Count == 2, + "a host an existing rule covers edits that rule instead of shadowing it"); + + // ------------------------------------------------------ 4. on the disk + SiteRules.Init(prefs); + Check(SiteRules.All.Count == 2, "the rules come back from the file"); + SiteRule spotify = SiteRules.For("https://open.spotify.com/"); + Check(spotify != null && spotify.Ua == 3 && spotify.Images == 1, + "with both fields intact"); + SiteRule accounts = SiteRules.For("https://accounts.spotify.com/"); + Check(accounts != null && accounts.Ua == SiteRule.Unset && accounts.Images == 0, + "and a field nobody set stays unset across a save"); + + // Forgetting is a real operation and says whether it did anything, so + // the screen can tell "cleared" from "there was nothing to clear". + Check(SiteRules.Forget("https://example.com/") == null, "forgetting an unremembered site says so"); + Check(SiteRules.Forget("https://open.spotify.com/") != null && SiteRules.All.Count == 1, + "forgetting a remembered one removes it"); + SiteRules.Init(prefs); + Check(SiteRules.All.Count == 1 && SiteRules.For("https://open.spotify.com/") == null, + "and the removal survives a restart"); + + // A file another build wrote, or one somebody edited by hand. None of + // it may stop the browser: an unreadable preference is a forgotten + // preference, never a launch that does not happen. + string junk = Path.Combine(root, "junk"); + Directory.CreateDirectory(junk); + File.WriteAllLines(Path.Combine(junk, "sites.tsv"), new[] + { + "example.com\tua=1\timages=0", + string.Empty, + "\tua=1", + "nothing.test", + "weird.test\tua=notanumber\timages=7\tfuture=1", + "UPPER.TEST\timages=1", + }); + DiagLog.Lines.Clear(); + SiteRules.Init(junk); + Check(SiteRules.All.Count == 2, + "blank, nameless and empty lines are dropped (" + SiteRules.All.Count + " kept)"); + Check(SiteRules.For("https://example.com/") != null && + SiteRules.For("https://upper.test/") != null, + "the two real ones survive, the second lower-cased"); + Check(SiteRules.For("https://weird.test/") == null, + "a line whose every field is unreadable is not a rule"); + + // ------------------------------------------------- 5. the other site + // + // Issue #75. History is most-recent-first, so the first entry that is + // not on this site is the site the user came from — and from there, the + // first that is not on *that* one is the site they left. One action + // therefore alternates between the two, which is how the question was + // asked. + var history = new List + { + new Bookmark("https://open.spotify.com/album/2", "Spotify"), + new Bookmark("https://open.spotify.com/album/1", "Spotify"), + new Bookmark("https://www.instagram.com/reels/x/", "Instagram"), + new Bookmark("https://www.instagram.com/", "Instagram"), + }; + + Bookmark other = SiteRules.OtherSite(history, "https://open.spotify.com/album/2"); + Check(other != null && other.Url == "https://www.instagram.com/reels/x/", + "from Spotify it opens the last Instagram page"); + Bookmark back = SiteRules.OtherSite(history, other.Url); + Check(back != null && back.Url == "https://open.spotify.com/album/2", + "and from there it comes back to the last Spotify page"); + Check(SiteRules.OtherSite(history, "https://i.instagram.com/api") != null && + SiteRules.OtherSite(history, "https://i.instagram.com/api").Url == + "https://open.spotify.com/album/2", + "another host of the same site counts as the same site"); + Bookmark fromHome = SiteRules.OtherSite(history, HomePage.BaseUrl); + Check(fromHome != null && fromHome.Url == "https://open.spotify.com/album/2", + "from the start screen it opens the most recent site"); + Check(SiteRules.OtherSite(new List(), "https://a.test/") == null, + "an empty history has nowhere to go"); + Check(SiteRules.OtherSite(new List + { + new Bookmark("https://a.test/2", "A"), + new Bookmark("https://a.test/1", "A"), + }, "https://a.test/2") == null, + "and neither has one that has only ever seen this site"); + Check(SiteRules.OtherSite(new List + { + new Bookmark(AsEngineUrl(""), "Overscan"), + new Bookmark("https://b.test/", "B"), + }, "https://a.test/").Url == "https://b.test/", + "a generated page in the history is skipped, not switched to"); + + Console.WriteLine(); + Console.WriteLine(_failures == 0 ? "siterules: all checks passed" : "siterules: FAILED (" + _failures + ")"); + return _failures == 0 ? 0 : 1; + } + } +} diff --git a/tools/siterules/run.sh b/tools/siterules/run.sh new file mode 100755 index 0000000..ac27a33 --- /dev/null +++ b/tools/siterules/run.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +# Exercises the per-site settings — SiteRules — off-device. +# +# ./run.sh +# +# Issue #74 asked for images and identity to be remembered per site, because +# Instagram wants the desktop identity and no images while Spotify will not play +# without them, and one switch for both means flipping it by hand on every +# crossing. Issue #75 asked for a way between two sites that is faster than +# typing an address, which is the same question about the same word: what counts +# as one site. +# +# That word is the whole risk. This file is in src/common, so it is in all six +# packages, and every way of getting "the same site" wrong is silent from a TV: +# too narrow and a rule stops applying the moment a site hands you to another of +# its own hosts, too wide and one site's settings land on a stranger's — a +# suffix test that lets notinstagram.com match instagram.com looks exactly like +# a rule that works, right up until it does not. +# +# This compiles the shipping src/common/SiteRules.cs against the Store and +# HomePage it leans on (with a stub for the diagnostics log) and holds it to: +# 1. the name a site is remembered under — www., case, port, credentials — and +# the addresses that are on no site at all, both shapes of this app's own +# start screen among them; +# 2. what a rule covers: the site, anything under it, and nothing that merely +# ends with the same letters; the most specific rule winning where two apply; +# 3. what a rule says: a field nobody set is not "off" — it follows the +# browser-wide switch — and a second press edits the covering rule rather +# than shadowing it; +# 4. the file: a round trip through the disk, forgetting that reports whether +# it did anything, and a hand-edited or future-build file that cannot stop +# the browser from starting; +# 5. the other site: that it alternates between the two sites somebody is going +# back and forth between, treats another host of the same site as the same +# site, skips this app's own generated pages, and has an answer when there +# is nowhere to go. +# Needs only the .NET 6 SDK under ~/.dotnet-local. +set -euo pipefail +cd "$(dirname "$0")" + +DOTNET="${DOTNET:-$HOME/.dotnet-local/dotnet}" +if [ ! -x "$DOTNET" ]; then + echo "siterules: no dotnet at $DOTNET (set DOTNET=...)" >&2 + exit 1 +fi + +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +cp Program.cs "$work/" +cat > "$work/siterules.csproj" < + + Exe + net6.0 + disable + Overscan + false + + + + + + + + +CSPROJ + +export DOTNET_CLI_TELEMETRY_OPTOUT=1 DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 DOTNET_NOLOGO=1 +DOTNET_ROOT="$(dirname "$DOTNET")" "$DOTNET" run --project "$work/siterules.csproj" -c Release -- "$work/data"