From Chaos to Clarity: My Journey with Obsidian

It's been a long time since I've written my last nerdy post. I will leave my entrepreneurial stories for when the chapter is closed, when there won't be a need to change names or the whole story.
As everything I tend to work on these days is ... classified ... let's talk about productivity.
Today I want to talk about Obsidian - and how it made me structured, or without exaggeration, how it changed my life.

The Need for a Tool

Every person is different. I am a born introvert force-turned extrovert-entrepreneur. It's easy for me to focus on one thing for 20 hours, but any context switching makes me extremely stressed. I am always fighting with a constant inflow of information with various signal-to-noise ratios - colleagues, prospects, customers, family, friends, email, Slack, browser, ....
When there are too many things, I simply cannot think straight. I need:

  • Clarity - what needs to be done so I achieve my goals, what is the next step?
  • 2nd brain - a place where I can store my thoughts to not occupy my working memory
  • Task system - something that will let me know if it's the right time to resume a project, handling all the complexities of priorities, deadlines, and dependencies without thinking about it

Requirements

My requirements for the tooling are as follows:

  • Supported platform - I mostly use my Linux-i3wm laptop and Android phone - it needs to work there
  • Synchronization - I want to access the tool from all my devices
  • Offline - I want to avoid situations where I am not able to access my notes
  • Encryption - I don't want to fear writing anything personal into the notes
  • Rich text - I want to write formulas, diagrams, and use pictures
  • Visual representation - I need to edit DAG describing what is needed to do next
  • Structure - I need to structure my notes in a tree with unlimited depth
  • Performance - I need the tool to be blazing fast, not waiting for page loads or rendering
  • Open format - I don't want my 2nd brain to be dependent on a proprietary format

It sounds pretty basic, right?
For years I've tried notion.so, doit.im, Microsoft Tasks, Google Keep, Evernote, and others.
They are amazing tools, but each of those had some downsides, usually significant ones and usually several of them.

My Workflow - Obsidian + SyncThing + NeuraCache

i3wm Scratchpad

When I hit $mod+o my Obsidian window appears/disappears - allowing me to put it on top of anything I currently work with. It's achieved through the "scratchpad" concept of i3wm - it's another workspace that is invisible and can host some windows. You can move windows to and from the scratchpad.

Scratchpad

Tasklist

I am using the right pane in Obsidian to permanently show my task list.
(You simply open the page and drag-and-drop it to the right pane - it will stay there.)
It's a plugin that indexes all the .md files and looks for the syntax of the task definition.
It allows you to set your deadlines with natural language (e.g. "next Sunday").

- [ ] take out the trash 🔁 every Sunday 📅 2021-04-25

Example: In the picture, you can see my notes about DES, where I created a note that I want to try implementing the Feistel Network in Rust to confirm my understanding.

I try not to remember what I am supposed to do, otherwise it occupies part of my mind, preventing me from thinking properly.
My workflow is simple:

If I need to do anything, I just write it down. The important thing is that the activity must be in your power to complete and should be rather atomic, otherwise it increases the stress level (waiting for someone to respond, etc....).
Just always write down what is the next step:

  • Think about next steps of project X
  • Tell person X about Y
  • Check that person X delivered Y as promised
  • Do X
  • Read X

I follow a simple rule that if I am done with the activity, I write follow-up tasks (if needed) ensuring continuity of the project.

Tasklist

Daily Notes

What helps me a lot is to have a new sheet of paper for each day - this is my base of operations - I jot down what I am working on and when I am interrupted, I can simply go back to where I was.

I use a simple template with:

  • Link to my TODO list (in case it's closed)
  • Scratchpad section (I open #h2 heading for each topic I work on)
  • Journal - states of mind, things that have happened to log, etc...

Whenever I work on some long-term project, I simply put a link from my daily note to the proper page.
The daily notes can also host the tasks created that day and can include some more context in case it takes you a long time to get back to the task.

Daily notes

Weekly Reviews

Unless you accept that the environment forms your identity or you are a specially rare breed, you need to consistently work on self-improvement and ensure you don't slip back to where you came from. What personally worked for me is to have a weekly review of ... of me.
I have a template that guides me through the review and of course, recurring Obsidian task to remind me.

I just challenge myself with questions that are just painful to answer if I am not on the right track :)

Weekly

Flashcards

As discovered by Hermann Ebbinghaus, people tend to gradually forget information, roughly following the so-called "forgetting curve."

  • Immediately after learning: 100% retention
  • 20 minutes after learning: 58% retention
  • 1 hour after learning: 44% retention
  • 1 day after learning: 33% retention
  • 1 week after learning: 25% retention
  • 1 month after learning: 21% retention
  • 1 year after learning: 15% retention

Forgetting curve

I refuse to accept this. As suggested by the "Unlimited Storage Hypothesis," the limits of storage are likely very high. The issue is just recall.

It's possible to hack this process by a technique called spaced repetition. If you review the information, it gets reinforced and you will reset the curve to its origin. I am using the app NeuraCache designed for spaced repetition. It scans some folder with .md files and looks for definitions in proper format. This allows me to create flashcards during my learning on desktop and I can review them on mobile. You are presented with a deck of flashcards that can optionally have two sides (question, answer). You try to answer it yourself, turn the card around, and answer if you knew it or not. Based on that, the cards are re-arranged, spacing the things you remember exponentially further in time while keeping the ones you don't recall on the top of the deck.

There are many formats for both questions (with answers) and spaced repetition (you want to convince yourself about something by repeating it). It can be as simple as:

max speed of a cat : 30 mph #flashcard
View in Obsidian View in NeuraCache
Flashcards definition Flashcards definition

DAG in Canvas for Clarity

As mentioned before, what helps me get clarity is to compress my universe into a Directed Acyclic Graph, where I describe the causality of my world. I want X, for that I need Y, for that Z. This together with a bit of stoicism is quite calming - you are just working on the leaf node/sink node/terminal nodes of your graph, knowing that you are doing everything that can be done to move towards your goals.

I apply this every time I feel overwhelmed: "So there is a lot of noise... what is important now?"

In Obsidian, there is a tool called Canvas, allowing you to create DAGs (and much more, it's like your private Miro).

Canvas

Installation and Configuration

i3wm

To bind shortcuts to bring Obsidian into/from the scratchpad, simply edit ~/.config/i3/config:

# OBSIDIAN
bindsym $mod+o [class="^obsidian" instance="^obsidian"] scratchpad show
for_window [class="^obsidian" instance="^obsidian"] move scratchpad, scratchpad show
bindsym $mod+Shift+o exec obsidian

Templates

Daily notes can be configured to be created from a template.
If you need templates, for example for meeting minutes or weekly reviews (of yourself), you need to enable them (it's a core plugin that's disabled by default).

Just set it in Settings -> Core Plugins -> Templates

Daily notes

Daily notes is a core plugin, that is disabled by default. Simply enable the plugin in Settings -> Core Plugins -> Daily notes. After enabling the plugin, just set the folder where new notes will appear and choose a template. You might also want to enable "open daily note on startup".

Setup daily notes

[Plugin] make.md

This plugin makes the experience much like with Notion.so.
It allows you to:

  • Open command palette with /
  • Assign icons to your folders
  • and more

Canvas

[Plugin] Tasks

Tasks is a community plugin, just install it in Settings -> Community Plugins -> Browser.
It works in a way that it indexes all your .md files and is looking for a specific syntax of tasks.
The tasks can then be queried by their query language.

I have a page Tasks overview that aggregates all the tasks found across all the .md files.

## Overdue

```tasks
not done
due before today
```

## High importance

```tasks
not done
priority is high
```

## <= Today
```tasks
not done
due today
```

## Tomorrow
```tasks
not done
due tomorrow
```

## < Friday
```tasks
not done
due on or before this Friday
sort by priority
```

## <= 7 days
```tasks
not done
due after today
due on or before in 7 days
sort by priority
```
## <= 30 days
```tasks
not done
due after 7 days
due on or before in one month
```
## > Next month
```tasks
not done
due after in one month
```

Synchronization

I use the tool Syncthing. It's a cross-platform tool (I've tried it on Linux, MacOS, and Android) that can synchronize arbitrary files.
The files are watched by the tool with Inotify/FSEvents together with periodic scanning.
Each device is paired with your other devices by exchanging certificates.
The file changes are then sent in encrypted form either as p2p (discovery through mDNS) or through discovery and relay servers as a fallback.

The actual setup is easy on all platforms, only hint is to start with the device you actually have your data, then add the neighbor devices (exchange certificates), which will offer you to share the folder with those devices. It will also work in the opposite direction, however, this is a bit more straightforward.

Syncthing