-
-
Notifications
You must be signed in to change notification settings - Fork 4
Guide: Atomic highlights
The Atomizer transforms your Readwise highlights and notes into individual, atomic files. Instead of keeping all highlights in a single document, highlights becomes a separate note file. It is an extremely versatile, powerful, and hopefully useful new function in the Readwise mirror plugin. Because it is so powerful and versatile, you should read this section carefully and make sure you understand the implications.
Warning
This function is new and therefore not as tested as other parts of the plugin. Expect things to break when enabling it. Before applying it to your entire vault, we recommend applying it to a few notes individually by using the "Update current note" command, as well as by using the conditional atomizing). Please also have a look at the Limitations and Caveats sections below. In particular, expect a severe performance hit when enabling the Atomizer and syncing a lot of highlights: a potentially massive amount of new files will be generated. Obsidian will have to process all these, their metadata, and links to and from these files. A full sync might possibly create tens of thousands of small new files.
When enabled, the atomizer:
- Parses your highlight template for special comments and extracts the content in those blocks as separate files
- Places embedded references (
![[filename]]) in the original document to create a deep web of linked notes (Optional, recommended) - Inherits the Frontmatter template of the parent note as basis for the atomic note's frontmatter (Optional)
- Maintains bi-directional links between parent note and atomic (highlight) notes by adding special properties to each atomic file that can be used to link back to the original note, and by exposing the linktext of the parent note for backlinking
- Allows the user to conditionally atomize notes by setting a special property (
rw-atomize) totrue(you can control what will be atomized with conditional atomizing)
The atomizer works like a second nunjucks template dialect with two available blocks: atomizer and frontmatter.
- With
atomizer, you encapsulate a template block into a separate file. It has three parameters,id,basename, andembed.idmust be set to the{{ id }}variable,basenameshould be used to construct the filename of the highlights (can simply be{{ id }}, or something more complicated like the following), andembedcontrols whether a link to the atom will be included in the parent node. - With
frontmatter, you can create dedicated frontmatter for the atomized file. This will overwrite the parent notes' values if frontmatter inheritance is enabled. We will always add the tracking property (default key:uri), and a new parent property (default key:rw-parent) which both link to the respectivereadwise_urlvalue of the note or a highlight.
Warning
The atomizer can break things. If you use it outside the scope of a highlight, unexpected things might happen. You should escape all variables used for values in your frontmatter key: value pairs with the fme filter: key: {{ variable | fme }}.
The atomizer opens a whole world of possibilities. The following examples are some simpler approaches, you can find some more advanced examples in the Guide with Advanced atomizer examples in the Wiki.
In it's simplest form, an atomizing highlight template looks like the following. This template will render the highlight ({{ text }}) and note ({{ note }}) fiels in one file per highlight, without any additional frontmatter:
%%! atomize
id={{ id }},
basename="hl-{{ doc.id }}-{{ id }}",
embed=true !%%
{{ text }}
{{ note }}
%%! endatomize !%%
The following is a more complex example which will add both the highlight and the note to the frontmatter (properly escaped), as well as render both in meaningful ways in the body of the note. It also includes a backlink to the parent note, based on the doc.linktext property.
%%! atomize
id={{ id }},
basename="hl-{{ doc.id }}-{{ id }}",
embed=true !%%
%%! frontmatter !%%
title: {{ doc.title | fme }} %%% This would render the title of the parent book %%%
type: {{ ["highlight", "foo:bar"] | fme }}
note: {{ note | qa | fme }}
highlight: {{ text | replace('__', '==') | fme }}
parent_note: "[[{{ doc.linktext }}]]" %%% This will include a backlink to the parent note in the frontmatter
%%! endfrontmatter !%%
{#- Highlight template that does a few things:
- Convert Readwise's "Highlight" syntax (__Highlight__) to Obsidian Highlight Syntax (==Highlight==)
- Link to the Source and to the Highlight in Readwise
- Add block-id's both for the highlight and the note (Please be aware that line breaks, lists etc. in a note might create multiple blocks and referencing the "block-id-note" block might only link to the last block of the note.
#}
{#- Example template using both filters #}
{%- if note | is_qa %}
{{ note | qa }}
^{{ id }}-note
**Original Highlight:**
> [!quote]
> {{ text | bq | replace('__', '==') }}
{%- if locationUrl %}
> (*Original*: [{{ location }}]({{ locationUrl }})){%- endif %}
{%- if location_url %}
> (*Original*: [{{ location }}]({{ location_url }})){%- endif %}
^{{ id }}
{%- else %}
> [!quote]
> {{ text | bq | replace('__', '==') }}
{%- if locationUrl %}
> (*Original*: [{{ location }}]({{ locationUrl }}))
{%- endif %}
{%- if location_url %}
> (*Original*: [{{ location }}]({{ location_url }})){%- endif %}
^{{ id }}
{%- if note %}
{{ note }}
^{{ id }}-note
{%- endif %}
{%- endif %}
%%! endatomize !%%
This template will create a file per highlight and a separte note file (if the note exists):
%%! atomize
id={{id}},
basename="hl-{{doc.id}}-{{id}}",
embed=true !%%
%%! frontmatter !%%
highlight:{{ text | replace('__', '==') | fme }}
parent_note: "[[{{ doc.linktext }}]]" %%% This will include a backlink to the parent note in the frontmatter
%%! endfrontmatter !%%
> [!quote]
> {{ text | bq | replace('__', '==') }}
{%- if locationUrl %}
> (*Original*: [{{ location }}]({{ locationUrl }}))
{%- endif %}
{%- if location_url %}
> (*Original*: [{{ location }}]({{ location_url }})){%- endif %}
^{{ id }}
({% if url %}[Source]({{ url }}), {% endif %}[Readwise]({{ readwise_url }}), [[{{ highlighted_at }}|{{ highlighted_at }}]])
%%! endatomize !%%
{#- This note will only be rendered as an atomized file if a note exists.
#}
{%- if note %}
%%! atomize
id={{id}},
basename="note-{{doc.id}}-{{id}}",
embed=true !%%
%%! frontmatter !%%
note: {{ note | qa | fme }}
parent_note: "[[{{ doc.linktext }}]]" %%% This will include a backlink to the parent note in the frontmatter
%%! endfrontmatter !%%
{%- if note | is_qa %}
{{ note | qa }}
^{{ id }}-note
{%- else %}
{{ note }}
{%- endif %}
({% if url %}[Source]({{ url }}), {% endif %}[Readwise]({{ readwise_url }}), [[{{ highlighted_at }}|{{ highlighted_at }}]])
%%! endatomize !%%
{% endif %}
Important
Enabling this feature will create many new files in your vault. Consider testing with a small subset first to understand the workflow and impact. You can also use the conditional atomizer (see below) to reduce the number of atomic files.
For large vaults, atomizing all your Readwise notes might neither be practical nor desirable (See Caveats). By enabling conditional atomizing in the settings, you can control which notes will be atomized and which ones not. This way, you can gradually build-up an atomic library. If enabled, you must set rw-atomize to true for an individual note. Once enabled, that specific note will be atomized the next time it is updated. If you set rw-atomize to true in the frontmatter, all new or updated notes will be atomized.
You can use templating logic to set the property dynamically: For example, if you use a block like the following in your frontmatter template, rw-atomize would be set whenever a note has the atomize tag set in Readwise, whereas all notes without the tag would not be atomized.
{%- if "atomize" in tags %}
rw-atomize: true
{% else %}
rw-atomize: false
{%- endif %}If you want to be able to toggle atomization based on a tag, and manually toggle atomization as well, you can simply remove the {% else %} block (note though that this will not allow you to manually disable atomization for notes whose rw-atomize property is set to true because of a tag):
{%- if "atomize" in tags %}
rw-atomize: true
{%- endif %}The following would atomize all Readwise docs in the article category:
{%- if category == "articles" %}
rw-atomize: true
{% else %}
rw-atomize: false
{%- endif %}More examples can be found under Advanced atomizer examples in the Wiki.
Warning
Atomic notes and highlights created this way will not be deleted automatically when you disable rw-atomize for a file, you have to manually clean up your Vault.
Contrary to the parent note, the atomized files are not (yet) tracked. Every run will completely overwrite any file created by the atomizer. If you change the basename template, new files with different names will be created, and links might break. Same is true for the doc.linktext to the parent note. Looking into this and potentially enabling tracking for atomized notes is on the Roadmap (without ETA).
The location of the files created by the atomizer can not be set, it is fixed to Highlights in the Readwise library.
When migrating to atomic notes, you should consider the following:
- Atomizing notes potentially creates a massive amount of new files (in the 10'000's for huge Readwise libraries). Depending on other features and plugins you have enabled, using the atomizing feature can considerably increase the time needed for a sync from Readwise. This is in particular true if you do a full-sync, but has less impact when doing an update sync or syncing an individual note. Features that might interfere are in particular those related to special handling of individual files: Enabling Obsidian sync for your Vault (or another online-sync solution, for that matter), having the Highlights folder and its contents being shown in the sidebar, using plugins that set icons for individual files, or (if you extensively use frontmatter in your highlight notes) plugins related to frontmatter (incl.
dataview,bases, and related ones). There's not much we can do about this in the plugin. You can try to exclude the Highlights folder if the plugin allows that and if it makes sense, and you can try the features that enable conditional atomizing (See Conditional atomizer). - If you use block embeds (
![[Readwise note^<id>]], where<id>is the value of a highlight'sidused in the highlight template), these links will most likely break. There are manual ways to deal with that, including the Persistent Links plugin for Obsidian which can fix broken links by finding the new file with the corresponding block-id. - Currently, we do not validate your Highlight template with test data. It is therefore your responsibility to creat a Highlight template which creates valid Markdown, including the Frontmatter.