Slopfluencer

Self-extensible software is here and its amazing

2026-06-02T00:00:00Z

Self-extensible software is here and its amazing

Source Material

This is an X article I published about self-extensible software.

We probably have a little bit of time before the all consuming machine gods take over our machines and everything is simulated. So until then, traditional software is probably going to stick around... right? Right!

Back in the dark ages, when code was expensive, one developer would build software for many people. These days, since code is cheap, everybody can build their own bespoke software? Software is truly dead right?

Well.. I guess the reports of software's demise are greatly exaggerated.

The truth is, people use prepackaged software because they don't want to build the dang thing from scratch, even if they could. I could build a house myself, if I wanted a shittier house at 3x the price that took 10x as long to deliver. I just want to buy a good house and customize it. The same goes for software.

Prepackaged software isn't going away. So does this mean the AI revolution is over? Pack it up? No, it means there's some truth in the middle. Self-extensible software is a happy medium.

We've had extensible software for as long as software has been around, but its always required a developer to build extensions for. Tons of software operate on this model, from dev tools to blogging platforms (Wordpress) to payment platforms (Shopify). People love extensibility. You can take a piece of software that handles 80-90% of what you need, then just adapt it for your own use case or how you want to use it. The only problem is that it used to cost a lot, you had to pay a costly developer for their time.

With coding agents though, you can now ship a developer INSIDE your software!

The developer is INSIDE the computer?!

While working on my self-extensible agent harness Neon Pilot (heavily inspired by the Pi coding agent), I've come to believe that self extensibility is the next core component of software. If you've used Pi before, there is a point where you tell your agent to create an extension for itself, and it just... does it, and it works. And then you get that "holy shit" feeling and you can't go back to regular software.

I'm going to talk about what I've built and what I've learned from building Neon Pilot and from @badlogicgames and Pi.

Making software self-extensible

The recipe for self-extensible software is relatively simple. You need:

A coding agent (inside your software)
A runtime
An extension API inside your existing software
Documentation for the agent

Neon Pilot is a coding harness, so a coding agent is already core functionality. We bundle Pi as the core agent harness and heavily extend it. For more traditional software, you can bundle one of the many available agent harness SDKs into your software.

The Pi coding harness and NP run on Node. You need an interpreted language in order for users to build extensions because you can't bundle a full developer tool chain or expect users to have all the necessary tools on their system. So you use an interpreted language and everybody's happy (...mostly...).

The real meat and potatoes of self-extensible software though is the extension API and documentation. The extension API defines the capabilities that the agent can use to build extensions, and the documentation provides the how. You need both, and this was one of the key insights from the Pi coding agent.

In Pi, right inside the agent's system prompt is a link to Pi's documentation. The agent is able to read documentation very easily and use that to build extensions. That's how it "just works". The agent is never guessing how to build extensions, its given everything it needs.

So this translates to self-extensible software as a capability layer exposed as the extension API, and a repository of documentation that the agent can read and draw from to build an extension.

The extension API is the next piece. To make extensions truly useful, you need to expose primitives of your application. The more raw the primitive, the wider the range of things the agent can build for the user. For example, if you expose a database primitive, then the agent can build dang near anything, whereas if you just expose a rigid API then the scope of extensions an agent can build is limited to the API.

It'll be up to the people building products to decide what level of customization they want their users to have. Users also have a say though, and if they want certain functionality, they'll also have the ability to hack it in or work around your system's limitations. So it's a push/pull.

Extensibility is good, customization is great

I fully expect more and more companies will augment their own applications with extensible surface areas. Big companies will struggle with this more. Its harder to turn a huge ship and it will be more difficult to build augment-able surfaces into their legacy systems.

Smaller companies that build extensible systems from scratch will be able to natively support this from inception, and this will be a huge boon to their business. What kind of business doesn't want to give customers the exact solution they want? It's always been a struggle to do this in the past. Its now possible, and those companies that embrace this will be able to provide a better product for users.

We're so early

If you've gotten this far, the message I want you to take away from this is that it is still very early days. We're in the TUI/console days of building self-extensible software and the solutions are only available to people developing software. We're still learning how to work with these new systems, and I fully expect more and more of these lessons to start to disseminate into the wider world.

I don't believe that some pie in the sky all consuming machine god that will take over our lives. That's a myth used to pump up company valuations since we simultaneously exist with two truths. We are in a bubble, and we are also experiencing a stepwise improvement in technology. The two truths can coexist. The realistic view is that the future lies somewhere in the middle, a world where software isn't dead, but we also unlock new capabilities that make the products better.

For me, that's self-extensible software.

Seeding Context in Personal Agent

2026-05-06T00:00:00Z

Seeding Context in Personal Agent

Source Material

This is an X article I published about Suggested Context in Personal Agent.

A new conversation with your agent is either a time for new beginnings, or an annoying arduous task of re-populating the agent's context. A situation I've encountered one too many times is needing to re-seed a conversation with context.

Pi's built in forking feature is amazing for this. You never have to start a new conversation, just fork off the context of an existing one. However you do still need to maintain a conversation and continuously rewind to a specific point in time.

What if you could take advantage of existing conversations and not worry about manually maintaining the conversation's context?

I built a feature into Personal Agent called Suggested Context.

Suggested Context does exactly that, suggests past conversations for your agent based on what you type into the input box, then lets your agent use the context or not. Maybe I should have named it "suggested past conversations" but alas its too late now, I'm already writing this post.

How it works

When you start a new conversation, Personal Agent searches your recent conversation history for threads that might be relevant to what you're working on. If it finds good candidates, it injects a small pointer block into the agent's context at the top of the prompt — before you've sent a single message.

Pointer block injected into your context, not full conversations!

The agent gets the title, conversation ID, workspace, relevance score, the signals that drove the score, and maybe a short cached preview from the conversation's generated summary. That's it. No full transcripts. No walls of old text. If the details matter for the current task, the agent is expected to call conversation_inspect on the conversation and poke around the transcript.

This strategy is the same that skills use, providing a minimal pointer for the agent to follow if it needs to know more.

The scoring

Scoring is intentionally simple. Each candidate conversation gets ranked on a small set of signals:

Title match: +5 per matched term, capped at +12
Same workspace/repo: +3
Summary or cached preview match: +2 per matched term, capped at +8
Transcript search index match: +1 per matched term, capped at +6
Recent activity (< 7 days): +2
Recent-ish activity (< 30 days): +1

Auto-surfaced candidates need to clear a minimum score of 6 to be included. The list is capped at 5 pointers, sorted by score, then by most recent activity.

I am very explicitly not doing semantic retrieval here. There's no embedding lookup, no cosine similarity, no vector database. Just stop-word filtered term matching against titles, workspace paths, generated summaries, and a pre-built search index. Similarity search is slow and expensive and requires chunking and embedding conversations.

The tool use side

When the agent actually wants to use a suggested conversation, it calls conversation_inspect. This tool lets your agent query the real transcript by searching blocks, filtering by role, reading windows around specific content.

This saves context by forcing the agent to probe conversation transcripts on its own rather than trying to generate relevant chunks of information like in more traditional RAG systems.

Does it work?

I just added telemetry so I need to collect more data, but initial results show it does get some use. Whether its actually useful or not is to be seen.

Personal Agent's Knowledge Base

2026-05-01T00:00:00Z

As I've been working on Personal Agent, one theory I've been operating under is that if I want an effective agent, it needs to be able to understand my intent.

Agents are just blank slates with default settings. The magic happens when it has enough context to be able to make decisions with the same understanding and principles that align with how I think.

Until models are actually able to internalize information, you'll need to manually curate this information somewhere. My answer to this is the knowledge base. A collection of information that both myself and the agent have access to and the freedom to edit.

Defining My Knowledge Base

My knowledge base is very simple:

Skills (repeatable workflows / runbooks)
Basic preferences and agent guidance (agents.md)
Freeform markdown files that the agent and I share

That's it. It's just a folder with markdown files. I've been a huge proponent of using YAML frontmatter with Markdown for knowledge management. I even created a note taking application that predated Obsidian that used this format.

Markdown is flexible and easy to read. The YAML frontmatter provides a place for metadata. That's really all you need.

Skills are a special subset of the knowledge base. They're repeatable workflows or runbooks that you want the agent to be able to reference, so they are injected into the context where the general knowledge isn't. The agent knows where the KB is located on disk and can explore and search it though.

Using the Knowledge Base

This is still an area I've been exploring. The current state of my KB is broken down into:

Wiki
Skills
Projects
Notes

Personal Agent has deep integrations with the knowledge base and the agent is aware of it and can explore and modify any part of it at will. It's easy to tag parts of the knowledge base in conversations with the agent.

Wiki

@karpathy's wiki idea was very cool and something I've tried implementing. In an ideal world, the agent would be able to index and form a web of information as my own personal wiki. That's useful for research and just giving the agent an understanding of the things that are interesting to me and within my sphere.

Skills

I consider skills just a subset of the knowledge base. They're repeatable workflows that are useful to inject into the agent's context so it can help itself.

Skills are just knowledge. This skill is my software development principles which are basically @GrugBrainedDev's: grugbrained.dev but rephrased so it doesn't accidentally bias the text output towards caveman speak.

This skill was one of the originating ideas of the knowledge base. It's not really a skill per-se because it's not really something I want the agent to execute, and it's not something I wanted to put in an Agents.md because not everything I want an agent to do is coding. It's more general knowledge.

Projects

I also use the knowledge base to store projects. It's extremely useful to have a location where I can dump anything related to a project. Then use that store of information to hydrate new conversations so a fresh agent can quickly ramp up on a project.

I often have ideas for a project that I want to explore but not implement yet. This is a great place to dump an idea, chat about it when it comes up, add more context, generally work on it over time.

I like storing communications with teammates about projects in here. There are usually decisions made in conversations that are useful context for myself and an Agent. Status updates on the project, notes about technical details (for example, keeping notes on the settings for running llama.cpp for my personal eval project).

Notes

Similarly for thoughts or ideas that don't have an umbrella, I have a separate notes folder to dump the ideas or thoughts.

I'm also now starting to explore offloading thoughts and ideas that I have a strong thesis on. The idea here is that if the agent has information about my beliefs, it is also useful guidance for it to make decisions in my stead.

Closing Thoughts

It does ultimately feel like a melding of minds. I try to offload myself into the system, which is shaped by the agent and also influences what I think.

There are a lot of knowledge systems that are basically overly complex RAG systems. We know agents are able to use simple tools like grep to explore code bases. RAG based retrieval systems might marginally improve the actual retrieval aspect, but nothing will populate the information into a useful format. It's the information IN the knowledge base that matters!

You can't offload teaching the agent either. You can try things like mining your past conversations for signals, but at best it can infer what you want. It can't read your mind, you need to teach it, and it takes effort.

Continuous learning, knowledge systems, etc are all very cool technology, but at the end of the day what makes an agent actually useful to you is if it's able to act for you, and in order to do that, it needs to know you.

Personal Agent Features (Part 2)

2026-04-26T00:00:00Z

Here are some more features in Personal Agent that you should steal to enhance your own agent harness.

If you haven't seen Part 1, check it out.

Reply with Selection

A small and somewhat insignificant feature, but very useful. Reply with selection allows you to highlight text from the transcript and inject it directly into the input box as quoted markdown. I use this feature a lot when I want to respond to certain parts of the agent's response.

New Conversation Context Seeding

Suggested Context

This feature is still being iterated on, but the idea is that sometimes I want to be able to seed a conversation with context from past sessions. This feature is on my new chat screen and executes a scored ranking algorithm on sessions within the last 2 days based on the text you type into the input box. It explicitly does NOT use semantic retrieval (RAG) to do this to avoid requiring an embedding model. Once you select up to 3 conversations, the conversations are compacted and attached to your new session.

The scoring mechanism:

The suggested-context ranking is a lightweight lexical search over recent conversations, not embeddings. It breaks your draft into meaningful terms, matches those against each conversation's title, workspace, transcript index, and generated summary, then weights title/phrase matches most heavily. It boosts conversations from the current workspace, conversations with summaries, unresolved/in-progress threads, and newer activity. A thread is only auto-selected when the top match is clearly dominant, so the UI suggests context without eagerly attaching random vaguely-related stuff.

Summarize & New

Allows you to compact a conversation and start a new session from that compacted summary.

Fork

Creates a new conversation up to a specific point in an existing conversation (lifted from Pi).

Parallel Execution

This is another experimental feature, think of it as Claude Code's /btw feature but beefed up.

Pi already has the ability to send a steering message to break into the agent loop at the next possible opportunity. Or Followup which queues a message to execute.

The parallel feature works like this:

PA forks the current conversation at the last stable point, so the side thread gets the same context without corrupting the main agent loop.
The side thread runs independently and tracks useful metadata like touched files, attachments, side effects, and possible overlap with work happening in the main thread.
When the side thread finishes, PA automatically injects the result back into the main conversation as a "Parallel response" once the parent thread is idle.
If the parent is still busy, the result waits in a small Parallel panel where you can import it, skip it, cancel it, or open the side thread directly.

SSH Remotes

Another experimental feature, SSH Remotes is a way for PA to SSH into another machine, install a small Pi and PA bundle and use that machine as a remote execution environment.

The desktop app is still the main interface, but being able to keep an agent loop running on another machine is pretty powerful.

This is still highly experimental and untested. I just haven't had a chance to use it much yet.

That's It for Now

I'm still adding features, but that about wraps up the main features I've added in the last few months of working on PA. I hope it's seeded some ideas for you while you build your own custom harness.

Thanks again to @badlogicgames for the wonderful Pi and the community here on X. I've gotten quite a few ideas from stuff I've seen on here.

Personal Agent

2026-04-25T00:00:00Z

I've been developing an agent harness I'm calling Personal Agent (naming is still hard, okay?). Personal Agent is built on top of @badlogicgames's excellent Pi coding agent. It's amazing what you can do with it and I've done quite a bit. PA is built on Electron and is 100% fully vibe coded. I've not looked at a single line of code.

The important thing is that I wanted a tool that's similar to the desktop harnesses coming from the big labs, but without big lab lock-in. With Pi, I can use any model and any harness. You need to own your own harness.

I'm not open sourcing it yet, because frankly this is for me, not for general consumption. However, I'd like to show you some interesting concepts I've built into Personal Agent that you can copy into your own tools.

Desktop UI

First and foremost, it's a desktop application because terminal UI is way too limiting. It's great for quick prototypes, but if you're trying to do anything more advanced you quickly need to start hacking in stuff like tmux which makes the whole thing feel like a hacked together kludge.

So, the first feature is a desktop UI that supports multiple threads. Heavily copied from @OpenAICodexCli. This allows me to easily swap between running conversations, see when conversations need my attention, and is the foundation for the rest of the features.

iOS Companion App

As an aside, I also have an iOS companion application. This companion app allows me to connect to a running instance of PA via my Tailscale network. This is really important as it allows me to walk away from my desk and monitor agent runs on the go.

Agent Lifecycle Management

I have a few features to improve long-running agent tasks and to put more control into the agent's hands. The goal is to give the agent tools so it can continue its own agentic loop.

Scheduled Tasks

The automation feature is powered by a scheduling engine that the agent also has access to. The agent can schedule tasks:

On a recurring basis
At a specific time in the future
Queued as a followup
Or as a deferred resume (retrigger the current loop in X minutes — syntactic sugar over option 2)

Deferred Resume

This has been a great feature so the agent doesn't have to abuse bash sleeps. It can just trigger a deferred resume in N minutes to wake its own agent loop up.

Scheduled tasks are exposed under the automations panel. It showcases active jobs and the threads they belong to.

Auto Mode

When auto mode is enabled, it injects a continuation prompt when the agent tries to exit, telling the agent to continue working. The review prompt says something like: continue if there is meaningful remaining work, stop only when the task is complete, blocked, or needs user input. Err toward continuing when useful work remains.

Then the continuation prompt fires: continue working on the current user request from where you left off.

Embedded Screenshot + Excalidraw

I wanted the ability to draw diagrams for the agent so I embedded Excalidraw. When you click the icon, it pops an Excalidraw modal, then attaches the image to your prompt. Very useful! You can also paste screenshots into Excalidraw and annotate them.

There is also a screenshot button that triggers Mac's native screenshot functionality, attaching it to the prompt for me.

Dictation

This feature currently uses OpenAI's dictation endpoint with my ChatGPT subscription. If I ever swap to something else I'd probably swap this out for a local Whisper model, but for now it's included with my ChatGPT sub.

Built-in Knowledge Base

I really wanted a way to combine my knowledge and the agent's knowledge. To do this, our shared understanding needs to be part of the tool. @karpathy showed us his cool Obsidian-based knowledge base, but I wanted it IN the harness. The knowledge base is simply a separate git repo, with a touchless sync engine.

I define "Knowledge" here as a combination of Skills + Agent.md + free-form documents/notes. I'm testing out a Karpathy-style wiki in here as well, but the real workhorse has been Projects + Notes.

I keep a folder of folders of projects, which both myself and the agent update and keep context. I also keep a folder of notes which are just markdown files where I can jot down ideas. It's great to just offload thoughts, ideas, todos.

The agent has full access to this folder — it has a pointer in its system prompt and can explore it. I can also use the @ symbol in my inputs to tag specific files in the knowledge base. All these combined allow me to more seamlessly integrate richer context into the application.

What's notably NOT in this is a RAG-based knowledge base. I experimented with RAG systems a lot both at work and with agents and frankly it's not worth the effort. It's simply easier and cleaner to just maintain a folder of markdown files so both myself and the agent can interact with it.

That's It for Now

There are so many more features in PA, but these are the main ones. You should copy these ideas if they resonate. You really should be building your own harness, owning your own agent.

FOBO, Grief & Bottlenecks

2026-02-14T00:00:00Z

FOBO & 5 Stages of Grief

Hooo boy. The last few weeks since my last post have been crazy. The software development world has been collectively going through the 5-stages of grief (and still is) while the tectonic shift in the industry is currently underway. Some people at the start of their grief cycle are still rejecting the notion that their jobs will change or still angry about it. Others are more in their depression and fear phase with "FOBO" or the Fear of Being Obsolete. Finally, there are people like me who work in the AI space that have had more time to process and are now in the acceptance phase and are now searching for answers for whats next.

Bottlenecks

So what do we know now?

Writing code isn't the bottleneck anymore

This might not be true for every aspect of software development at the moment, but it will be very soon.

Running computer systems won't be the bottleneck anymore soon

By running systems, I mean that writing code is solved and the next logical step is getting that softare in the hands of people. So creating and managing infrastructure will soon be automated. I'm already seeing people using coding agents at work to help resolve incidents and spin up infrastructure.

Agents are starting to move into non-software work

Claude Cowork is the first large deployment of ai-agents for general work.

There's also Claude in Excel which will give non-developers their first real "oh shit" moment with AI.

OpenClaw has caught the world by storm as one of the fastest growning open source projects. Its a personal assistant agent that has really pushed the boundaries (at the expense of safety) of what AI Agents can do. It's been interesting to watch.

What does this mean?

There's a lot of FOBO out there, but time and time again increases in productivity don't generally equate to mass unemployment and poverty. People find things to do and society doesn't really work if everybody is just milling around with nothing to do. This just means that work shifts elsewhere.

We don't still pick grain crops by hand anymore.
We don't mine coal with picks and shovels anymore.
We don't need typists to write memos with typewriters anymore.
We don't need human computers to crunch numbers anymore.
We don't need to mail correspondence to people anymore.
We don't need to go to the library to look up information anymore.
We don't need to go to the store to buy a record to listen to music anymore.

Instead we now have more leisure time than ever in history, food abundance, medical care, air-conditioning, running water, heated homes, the list goes on and on. And we're still the bottleneck.

Writing software is done, but we're not done.

So back to software. Writing software isn't the bottleneck anymore, but writing software was never really the job. Like most people in an organization, developers solve problems.

Problem -> Solution
A CEO needs a product to make money -> Delegate problem to product org
The product org needs to build a product -> Delegate problem to product and engineering
The product manager figures out what users want -> Works with engineering to build it

Before: Engineering builds product -> Engineers research, experiment, build POCs, build the software, QA it, launch it, iterate
After: Engineering builds product -> Engineers research, experiment, build POCs, build the software (now delegated to agents), QA it, launch it, iterate

What is happening here is that parts of our work are now getting automated. It means that the iteration cycle is faster and tighter, and that our productive capacity per engineer increases.

Thorsten Ball on twitter just posted a video about how humans are still the bottleneck. How for product development, we can see velocity shifts by reducing the number of layers between a user and the software. I think we'll see more of this to come to some extent.

However, the reason you don't see radical innovation at large companies is just due to their sheer bulk. There are layers upon layers of humans that form an organization, each increasing the amount of friction it takes to navigate through the layers. A lot of those layers are there for good reasons, especially at scale. Organizations are complex systems. We used to have large companies with pools of secretaries, typists, phone operators, mailroom clerks, etc. Many ancillary roles that were eventually automated away, yet companies are still as large and complex as ever. Humans have an innate need to form clans to solve problems. I don't think AI will change this.

I think ultimately the work of a developer will shift towards broader work, working across more of the stack where people were once specialized in an area, now they might specialize in multiple, for example, product + fullstack, or design + fullstack, or backend + infra. This also means that companies will be able to do more with fewer developers, which might be a good thing, or it might just means they'd be paying the same for developer + ai tool as they would for 2 developers. In that case, AI might not necessarily win out over having somebody you can trust and that's accountable working for you.

Parting words

Its a scary time, but not because there is an actual looming threat of AI. The ambiguity of the situation, the unknown unknowns is whats scariest. We don't know what to expect so we can't react, so we expect the absolute worst. But life goes on, generations come and go, we're still here and I believe that our human spirit will endure.

The (coding) vibes have shifted

2026-01-06T00:00:00Z

Its 2026 and the coding vibes have definitely shifted

As I wrote in Its Time To Let The Model Drive, the latest generation of models have hit some invisible line where they went from "meh, they're okay" to "holy crap they're really good". Over the break you could see this unfolding as bored devs off from work finally decided to check out what all the buzz was about and use these models to build stuff.

If you go on twitter, the vibes have definitely shifted. People are completely shocked at how good they've gotten. Particularly Opus 4.5 and GPT 5.2 Codex have been blowing minds.

Its caused quite the divide in the developer community with some on the vibe coding bandwagon, some begrudgingly starting to acknowledge that this is the future, and some staunchly against it.

But its pretty clear that vibe coding is here to stay and that coding agents are a force multiplier.

Joining the flock

Over the break I too joined the flock of developers working on side projects. I spent $500 in subscriptions and API credits building Familiar Fleet. I averaged 5-6 hours a sleep per night, the rest of the time was spent managing multiple agents building out various features in Fleet. It's been a long time since I've felt this compulsion to build something, and I haven't had this much fun in a long time.

I maxed out a $200 Cursor Ultra plan in 1 week pushing out 2.2B tokens.

I went into this project with only 1 rule. This is going to be a fully vibe coded app. I would not write any code.

So far, it's been working pretty well.

Fleet is a desktop app built with Rust (Tauri) + Typescript + Tailwind + Golang (backend). Its an agent orchestration system meant for non-developers and provides a nice interface to spin up agents. I'm actually not sure who its for, but I think the agent paradigm is very powerful and I think non-developers can get a lot of value from it, so I want to put this into the world.

Product Development

I went into building fleet with a really basic idea. "Claude code for normies". The hardest part about building something so vague is that you have no idea what works. What does it even mean? What do normal people even want? Are agents even useful to non-developers? Who are the people that would benefit from it?

The thing I really love about vibe coding is that it gives you the ability to iterate on the product very quickly. You can change something within minutes that might have taken me hours or days in the past to code. The ability to touch, feel, and interact with the thing I'm building makes all the difference when exploring a new space.

This is an important distinction that people online keep missing and arguing past each other about.

If your priority is to build stable, high quality systems, you probably don't want to hands-off vibe code your application.

However, if you're working on a startup that doesn't have product market fit, the ability to vibe code something and get it in front of people very quickly means that you get that many more chances to refine your ideas and build something that fits user's needs. It also means you get many more chances to get it right before you run out of resources.

Where do we go from here?

Ultimately, there is going to be more software in the world. Some will be good, some will be bad. The barrier to entry to develop is lower than ever. That's probably a good thing, as more people will be able to build and produce value. It also means change in the world, and in our industry. It's just a fact of life.

The Quantization Conspiracy Theory

2025-12-17T00:00:00Z

There's a ridiculous theory floating around on places like Reddit and Twitter that greedy model companies are selectively dumbing down their models for some profit motive. Typically these conspiracy theories focus on models getting dumber over time, or during peak periods, and the way model companies are doing this is through "quantization".

Nevermind the novelty factor of the models wears off over time, no, its the greedy companies trying to extract more profit from users!

Let's talk about this.

What is quantization

Quantization^[1] is the process of truncating model weights to obtain a smaller (and consequently faster running) model. Typically quantization will trade off model intelligence for a reduction in utilization of expensive gpu memory and also require less computation during inference. You can take a model that's been trained with FP16 or FP32 (Floating Point 16/32 Bits), chop off some bits and the models will continue to work pretty well. If you're interested in a deeper dive on model inference, check out this blog post in model inference speed of light calculations^[2].

The arguments

Some redditors discuss chat gpt quality issues

Hacker news commentor

Twitter user with subsequent response from an OpenAI employee

Users on Reddit complain about claude code

Are model companies quantizing their models?

Yes, most likely! It's an easy way to get performance gains from a model you already have, improving inference speed and reducing the amount of GPU resources you need for inference.

Are model companies quantizing their models after release?

Their models, no. Their products, maybe.

You need to break down model providers into two groups:

Big labs (OpenAI, Anthropic, Google, Mistral, etc)
Open source model hosts (Together.ai, fireworks.ai, etc)

This post will only cover big labs, not open source model hosts. Most of the complaints revolve around OpenAI and Anthropic.

The big labs typically offer a chat product (including coding products) and an API. Each are tailored to different use cases. Chat products like ChatGPT are targeted towards your average consumer, while APIs are typically targeted to business customers and developers.

APIs

A company like OpenAI typically releases a model under a name like gpt-5.2 and will typically also release specific versions of models like gpt-5.2-2025-12-11. The expectation is that model names like gpt-5.2 will be updated as they improve the model, whereas specific model versions like gpt-5.2-2025-12-11 won't change (and will be deprecated faster).

If OpenAI decided to try to pull a fast one over on their customers by "rug pulling" model quality on their API after release, their customers would find out almost immediately.

Consumers of these APIs are usually building products with the models. When you're building a product that uses a black box like a LLM, and you want to ensure the quality of your product is stable, you build an evaluation^[3] dataset and run it against your product anytime you change something. With all these customers running evaluations against these models constantly, its impossible that a performance impact will go unnoticed. Any attempts to sneakily implement cost cutting measures will be caught red-handed. A bump in model version could affect evaluation scores, and OpenAI has provided guidance in the past to pin your model version.

The other sure sign would be a sudden and abrupt increase of model response speed across the board. If you change the size of the model that you're serving, you will change the inference speed of the model output. If you take a large model, then cut the weights in half by quantization, you would see a corresponding speed increase due to the lower computation cost. This would be a consistent improvement, but also only tangential evidence. You could also improve inference speed by optimization to the inference software stack (like kernel fusion ^[4]).

"Oh they could be tricking you somehow by quantizing it and then just adjusting the speed they return the tokens to you!" -> That's seriously ridiculous. They're going to invest the time and infrastructure to build out some kind of system that will drip feed you tokens after? Seriously? Could you build a system to do this? Sure, probably. Would they do it? No. These companies leak like a sieve. If they tried doing that, it would be public pretty quickly.

So let's put API offerings aside and assume that the big labs won't mess with their biggest source of revenue.

Chat Products & Agents

So that leaves the products including chat bots and agents like codex and claude code. Are the companies sneakily quantizing models here?

They probably aren't dynamically quantizing models, but they may change the models used for the experiences!

ChatGPT rolled out their model router with ChatGPT5 which people hated. This router was able to dynamically allocate a user's request to different models. I have no doubt that OpenAI built in measures into this system to be able to dynamically control the flow of traffic to different models, especially with all the free user traffic they receive.

Does this mean that OpenAI was sneakily taking a model then quantizing it after users get used to the powerful version while rubbing their sneaky hands together?

No, however it does mean that they're able to dynamically scale up or down different models and route traffic to them based on current system load. This could potentially result in a different user experience for users that get routed to different models. Also, just because a user is routed to a different model doesn't mean they're getting a "quantized" model. There are many different ways to reduce inference cost including just using a model with fewer parameters, or different architectures (sparser mixture of expert models), speculative decoding, etc. Quantization is just one tool in a researcher's toolkit to reduce resource usage.

Another complaint is that Claude Code gets dumber at certain times. Anthropic did confirm an incident where there were several issues that caused a regression, so that's a possible explanation during that time. Another potential explanation could be context overload^[5] where the model's context fills up which the model cannot handle well (even though they say the context is of a certain length).

Intent

The general sentiment around AI companies has been prety sour. I get it, they promised a lot of bullshit and its a really bad look. However, I don't think they're trying to screw over Joe Schmoe by rug pulling them with a dumber model after release. I also specifically think that this idea that these companies are using quantization as the only tool in their belt to reduce GPU usage is ridiculous. If it smells like a conspiracy theory, sounds like a conspiracy theory, then it probably is a conspiracy theory.

What I do think is happening is two-fold:

The technology becomes normalized. People just get used to the thing and the novelty factor wears off. They now start seeing the flaws and are disappointed.
The companies are trying to optimize their product to be more resource efficient trading off product quality when it makes sense. I don't actually think this is a bad thing either, it's engineering. There is always a tradeoff and you do the best you can to provide the best product you can. I don't think they're doing it by sneaking around rug pulling users though.

What I'm pretty sure isn't happening:

The big labs aren't releasing a model then heavily applying quantization after-the-fact. They spend a lot of time and energy training the model, then evaluating their checkpoints to pick the one that performs the best. This process usually includes other inference optimization techniques they may apply to achieve the desired traits from the model (like quantization, speculative decoding, or other inference time improvements). They then release the model, and it goes out into the world as a version. They may make updates to the model, additional training or fixing issues that arise. They most likely release these as new versions of under the same model banner.
They aren't sitting around rubbing their hands trying to screw you out of your tokens. They have a high demand product, limited resources, and are just trying to build a good product for the most part.
Quantization isn't the only tool they have for model performance. Anytime discourse around model quality comes up, "quantization" always comes up like it's some big gotcha. By throwing around quantization, you're really not helping your case! If there are model quality issues, then talk about the model quality instead of coming up with conspiracies about how they're stealing your tokens.

Why did I write this?

A comment I received

I've been seeing this discourse for a while now and its been bothering me. I get the frustration when you use a product and you feel like its getting worse over time. I think its human nature to want to speculate about why it happens, and I just so happen to be a bit of an expert in this area, so it rubs me the wrong way.

Three things to take away from this:

Yes, there could be quality issues with these products. Regressions do happen, and these companies are constantly changing things an optimizing to improve resource utilization.
No, they aren't doing this to screw you over or for some profit motive (yet), and they aren't applying quantization to a new model after-the-fact to bait and switch you.
Quantization isn't just some hammer to bash models with, these are large, complex systems and other things could also affect product quality.

Footnotes

For a deeper dive into quantization, I recommend reading this page on hugging face ↩︎
https://zeux.io/2024/03/15/llm-inference-sol/ ↩︎
Evaluations are test sets where you keep example inputs and outputs and you test your system against to ensure quality has not changed when you make changes to your system. Anybody seriously building products will have evaluations, it's a fact of life when trying to maintain a consistent product. ↩︎
Kernel fusion is a technique that optimizes the software running on the GPU that takes into account GPU architecture to run calculations more efficiently. See Andrej Karpathy's video for a good explanation of kernel fusion ↩︎
Dex Horthy's talk on context management is very interesting. A rule of thumb is that models get dumb when around half their context fills up: https://www.youtube.com/watch?v=rmvDxxNubIg ↩︎

Its Time To Let The Model Drive

2025-12-12T00:00:00Z

Its Time To Let The Model Drive (for coding)

Its 2025 and you're still writing code by hand?!

Just a few months ago, I would not have trusted a LLM to generate tons of code. The quality of the models were good, but there was some intersection between model capability, my skill issues, and my engrained preference to control the code.

Something changed very recently. With Opus-4.5, GPT-5.2, and Composer-1, after using these models for a few weeks inside Cursor, I'm a full convert. The quality of the code is good (probably better than what I can write), the instruction following of the models is good, and I've discovered a workflow that really feels powerful.

It started when I attened the 2025 AI Engineer Summit which was focused on coding agents. Numerous talks covered topics like context engineering, Research - Plan - Implement workflows, using markdown files and skills etc. It opened my eyes to the reality that I needed to change how I worked and put more trust into the models.

The workflow

The workflow itself is pretty simple.

I use the smartest model I can get, then use it to sketch a plan.
- In the initial prompt, I often reference files that I know the model should use to provide more context, and I also try to give it as much context as I can.
- I use plan mode in Cursor which generates a markdown file with the implementation details.
- I then review the plan, and iterate on the plan with the model until I'm satisfied.
Then I implement the plan, with a fast model.
- In Cursor, I use composer-1. You can also use a model like Sonnet-4.5 to execute the plan.
Afterwards review the code and accept.
Have the model add tests.

The key point is something I learned in Dex Horthy's talk^[1], its important to create a plan and review it in plain english so you can catch potential issues before they're turned into code. This makes a world of difference!

I'm also a fan of cursor. I've used VSCode for years and its familiar and does what I want. You can do the same thing with CLI coding agents like claude code as well. I just hate reviewing diffs in the terminal interface.

Code organization

Another discovery is how you organize your code matters a lot in the age of agents. You need to spend extra time organizing your code into discrete, well organized chunks that the model can take and digest easily and that won't eat up their context.

What I mean is organizing your code in a way that you can tell the model to "go use the foo utility/package/library" and it can find the code easily, get context about it, and then use it. If you have a spaghetti mess, it will be very expensive for a model to poke around the codebase to learn how to use it.

Agent context

Using something like claude.md/agents.md is important. This document provides high level contextual information for your codebase that the model can easily discover and learn. Use a model to generate it, giving it stylistic guidance, contextual information, or other details that are important to point out.

Skills are another pattern that's starting to emerge. Skills are just folders with markdown files, scripts, or other files that a model can use to "learn" how to do a thing. It's just a folder with different chunks of context it can dynamically load if it needs to.

This is actually a really powerful pattern. When i mentioned above that you can tell a model to just use a utility/package/library, you're effectively telling the model to go use a "skill". It helps if your skills contain contextual data in the form of a readme, or you can follow Anthropic's skill spec. A skill is a neatly packaged combination of code + context and the model can use this discrete chunk to fulfill your intent.

Trust

I think letting go and trusting the models to do the right thing is hard. You need to continuously prove to yourself that you're giving the model the right context, and the model is doing what you asked. It's like you and the models are a team now, and like all teams, it takes time and effort to build a level of trust and understanding. But once you're on a high functioning, high trust team, you're able to get so much done.

It takes time and you'll need practice, but after a while it just clicks and its hard to go back. You may have to take the reigns sometimes and write actual code to get stuff done, especially with edge cases. I'm quite fortunate my employer is very AI-forward and provides a ton of support when it comes to trying the best tools and this freedom has allowed me to gain a lot of competency with the tools.

You'll need to practice, and you really can't cheap out by running open models (unfortunately).

Async vs Sync

There's also uncertainty between how asyncronous agents and syncronous agents should be used. Should you be firing off a bunch of tasks and then context switching between them? Should you just do one thing at a time with an IDE-like experience?

I think you can do both, but you need to be mindful about how you're using them.

Async agents

I have used an asyncronous agent at work that's able to run code against our own internal codebases. These are really powerful tools, but because they're asyncronous, the expectation is that you fire off the tool and it just runs and you check in at the end.

This puts the burden on you to context-dump at the beginning and put in a lot of effort thinking through all the potential context the agent might need. There's no real iteration loop. The expectation is you give it a task and it gets the work done.

This means that small, self contained tasks are perfect for async agents. Bug fixes, small JIRA tickets, anything that has a definitive answer for whether it's done or not, you can use an async agent. Just fire it off, it can run tests and once its done it'll create a PR for you. Easy.

Where async agents fail is if the task requires a human in a loop.

Syncronous agents

A syncronous agent runs in an IDE or in the terminal and you're continuously prompting it to work on small discrete tasks that add up to something bigger or more ambiguous. These agents require the driver to highly aware of the context around the code, and is more of a pair-programmer than anything. I've found the most success using the Plan/Research, then Execute workflow on small chunks of work.

An example:

Goal: I want to refactor a part of the codebase.
Chunk 1: Go find this module, move it over here, fix imports and references to this code.
Chunk 2: Split up this code into these separate modules, fix imports and references, add tests.
Chunk 3: ...so on and so forth.

The project is completed by implementing small discrete steps that you're able to plan, execute, and verify along the way. If you try to do a huge big bang refactor you're going to more than likely not like what you see. There's a lot of small decisions along the way that you as the human will need to make that if left up to an agent, may make all the wrong decisions (because they lack the context!)

Conclusion

Building trust and coming to a workflow that works for you is going to take time and a lot of tokens. Those of us that can adapt will implement more than ever before, and those that refuse will fall behind and become obsolete. It's unfortunate, but these tools are here to stay, they're far too economically useful.

Footnotes

No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer ↩︎

There's so much alpha in being terminally online

2025-12-12T00:00:00Z

There is so much alpha [for your profession] in being terminally online

As a beginner

When you're learning something new, it's so important to fully immerse yourself in whatever you can about that particular thing. Passive learning contributes a lot to becoming an expert and being terminally online is just increasing the amount of passive learning.

There were numerous times in my life when I've jumped head first into different areas and I've learned that just studying the subject isn't enough. You need to passively integrate whatever it is into your life.

As an example.

I'm a self taught developer.
I learned to code by doing and by reading books and online content.
I also learned by replacing my online reading and video content with programming related content. Switched to programming subreddits, started frequenting hacker news, etc.
I also attended a few events.

There's only so many hours in the day to do hard, strenuous, mental work. You eventually get fatigued, but there's still marginal gains from consuming related content.

It's like learning a language. If you fully immerse youself in a foreign country, you'll learn a language much more deeply and faster than if you just do Duolingo every day for the same amount of time. The depth of immersion is just not the same. You're essentially reprogramming your brain.

As a professional

Past the initial learning stage, being aware of cutting edge news in your area of expertise gives you first mover advantage.

You can take advantage of this at work, by being the first to share news and start discussions. Over time people will associate you with expertise in a particular area.

Being aware and passively consuming news and other information in your area of expertise also puts tidbits of information deep into the back of your mind. As you work on other things, this information is then useful to fill in gaps, give you ideas, or help you solve problems. There have been so many times I've suggested something that solved a problem because I had passively heard about a solution. It becomes a super power.

There's also being in-the-know for the terminally online in-group. References could be a wink to others that you're also part of the same community. There's nothing like seeing an obscure in-group meme in an out-of-context place and then getting excited you got the reference. That'll automatically warm you up towards the other person since you now have this shared understanding.

Pitfalls

Spending your time getting better at your craft is great. There's a lot of slop out there that tries to steal your time, giving you dopamine hits but nothing else in return. Replacing the garbage with something that's at least somewhat valuable is a much better use of your time.

You gotta watch out for the dumb games people like to play though. Politics, attention stealing controversy, slop-peddlers. In every online group the same things always happen. Somebody says or does something, other people take offense, flame wars ensue, feelings are hurt.

It's all so tiring and you'll do well to avoid it.

You also need to watch out for being too deeply invested. If you let it take over your personality or become your identity ("I'm a redditor" or "I'm in twitter t-pot"), then you're heading way into the cringe territory and you start getting entangled in the bullshit.

Recap

Information is good and being tapped into the information stream can be beneficial.
It's better to spend your time learning about something useful than consuming slop, even if it's low quality information.
Avoid the dumb bullshit games that people like to play in every group.

Task Oriented Dialog Systems

2024-10-04T00:00:00Z

Introduction

During my time at Clockwise, I worked on Prism, a calendar management product that leverages AI to interpret user commands to execute actions on their calendar. I won't go too deep into my work at Clockwise, but I will explore the concept of using domain-specific languages with LLMs to achieve a goal.

I stumbled across this concept when I found Grammar Prompting for Domain-Specific Language
Generation with Large Language Models. During the last year, I haven't seen much discourse on domain-specific languages in conjunction with large language models, and I've been trying to wrap my head around why. I'm not an expert in machine learning or natural language processing (NLP), so I have incomplete knowledge.

This post is a collection of insights I've gathered over the past year. If you're well-versed in Task-Oriented Dialog (TOD) systems, I'd love to connect and learn more from you!

Task-oriented dialog systems

From what I've gathered, the approach we used is a subset of NLP known as Task-Oriented Dialog (TOD) Systems.

What is a Task-Oriented Dialog (TOD) System?

From my understanding, a TOD:

Is a conversational system that helps users do something (achieve a goal).
Interprets the user's intent through baked-in assumptions or through workflows to disambiguate requests.
Manages the conversation state and has logic to complete workflows (to achieve a goal).

In a nutshell, it's a system that interprets a user's intent (their utterance), resolves ambiguity, and then formulates a plan for execution.

Fun Fact: In NLP, they use the term utterance to refer to a single unit of human speech or text. It's the basic unit of communication in NLP systems.

What's an example?

Take, for example, an assistant like Amazon's Alexa or Apple's Siri. A user may provide the following utterance:

User: Siri, what's the weather today?

Assistant: The weather today in New York is 74 degrees Fahrenheit.

How does the system go from the user asking what the weather is to it responding with the weather? This Task-Oriented Dialog system requires several major components:

Figure out what the user is trying to achieve: "The user would like to know the weather"
Remove ambiguity, such as the user's location: "The user would like to know the weather in their location"
Create a plan to execute: "Get the weather in their location and return that to the user"

Even breaking down this system into smaller pieces results in quite a bit of complexity!

The system needs a way to turn the user's request into a plan (a program).
The system needs a way to figure out the user's location.
The system needs a way to output this result to the user.

In order to get a machine to interpret the user's intent, the intent must be translated into a machine-readable plan. This is where a representation of complex ideas comes into play (a domain-specific language maybe?):

For example, what if our plan was a Python program?

def get_user_location():
  '''Gets the user location from previously stored user information or via device gps'''

def get_weather(location):
  '''Query an external weather API to get the weather in the user's location'''

def program():
  return get_weather(get_user_location())

That's a lot of different systems for a relatively "simple" request!

Why aren't they more popular?

This is a question I've asked myself time and time again over the last year. The approach is fairly solid, and with large language models, NLP is so much easier than it used to be. So why isn't it more popular?

Anatomy of a basic TOD system

First, let's look at what we need for a basic TOD system:

We need a way to turn a user's utterance into some kind of structured data that we can then interpret.
We need a way to train some model to output this structured data.
We need to then take the output plan and "run" it on our systems to execute actions to achieve a goal.

How this looks in the system we built at Clockwise was:

We generated user utterances and domain-specific language conversations synthetically using a larger model.
We fine-tuned an open-source model using this data.
We built an interpreter that would "run" this DSL and execute actions to achieve the user's goal and connected it to existing internal APIs.

It was a ton of work! It took me and a team of backend engineers a year to get a stable working system!

Downsides of TOD systems

So why aren't TOD systems more popular in this age of a massive influx of AI companies? In my opinion:

They just don't know about it.

Many people, myself included, enter the space with little to no prior knowledge or experience in NLP systems. We became interested with the popularization of LLMs and wanted to build things with this new LLM technology.

TODs are complex and hard to build!

It takes a lot of work to build a TOD system. Just the basic system I outlined required:

Creating a domain-specific language
Creating an interpreter for this custom domain-specific language
Generating a dataset to teach a model how to output this domain-specific language
Making sure this dataset was clean and didn't have issues like hallucinated keywords, or syntax errors
Doing the actual model training and tweaking hyperparameters
Serving the model on some kind of GPU-enabled infrastructure
Creating an eval dataset and setting up a way to test model output to control quality

It's a lot of complex components and takes a long time even if you have a team of highly experienced engineers.

It takes a lot of work to create custom datasets to train a model on your domain-specific language.

If you create a custom DSL, any trained model will have no context on that language since it doesn't appear in its training data. That means that you need to provide documentation and examples for how to use the DSL, and in turn, that means you need to have a deep understanding of the different use cases and appropriate actions the model can take.

Upsides of a TOD system

Control

With a TOD system, it is much easier to control the LLM output, constraining it to whatever is do-able by your domain-specific language. You can also keep your business logic out of the prompt for the model. Essentially your "plans" or programs in your DSL is your business logic and it can be expressed in some intermediate representation.

Auditability

With some intermediate representation of the user's intent, you can easily audit this plan or present it to the user for confirmation prior to executing the actions. This is very important to ensure the user experience is both reproducible and controllable. You don't want an LLM making decisions in some black box that you can't debug or explain to the user why it did what it did. You certainly don't want to let an LLM make potentially damaging changes to users' data without their explicit confirmation!

LLMs are not AGI

I think it's important to re-iterate that LLMs are not general intelligence. Leaving the ability for an LLM to make decisions is akin to letting users roll a weighted die on how to achieve their goal. It may work most of the time, but it can also blow up in their faces.

So why aren't they more popular?

Ultimately, I think they're just hard to build. Large companies have the resources to invest into NLP experts and teams of developers to build out complicated systems and main them. Smaller companies that want to "implement AI" are just trying to find low hanging fruit and build their entire workflows around LLMs.

There is also a fatigue around "chat bots" happening now. Lots of chat focused applications are thin wrappers over LLMs and they've shown their functionality to be quite limited. There has been a push to integrate LLMs in other ways into products like copilots or agents.

If you have a different theory, I would love to hear it though!

Alternative: Function Calling / Tool Use

For those of us coming into the NLP and TOD space from the outside, we encounter function calling or tool use. Function calling is a popular feature implemented in many large LLM services that allow LLMs to "use tools" or reach out to external APIs or to call code.

These are good tools if you're using an LLM as a conversational partner and to enable the LLM to achieve more, but for a TOD system, function calling requires embedding all your business logic into LLM prompts. Ultimately, this means less controllability of a highly non-deterministic system, and ultimately decision-making is left to the LLM. Letting an LLM control your business logic is not good! You want full control and accountability of your system!

Just use Python

Why create your own language when you can use an existing language like Python? The model already knows how to write Python.

Yes, that's true. We looked into this and you run into two main problems.

First, you need to figure out how to sandbox Python and be sure there are no leaks anywhere; otherwise, you create a major security hole.
Second, you run into the issue of the model trying to use its vast knowledge of Python to do all sorts of stuff.

When we experimented with Python output, the model tended to try to import packages, both from the core library and third party. This opens up another vector for attack.

Finally, Python is fairly verbose, and the more verbose your language, the more tokens it takes to generate an output, the slower your overall system will be.

Isn't fine-tuning dead?

Why are you fine-tuning models? Didn't you know that fine-tuning is dead?

Well, I'm sorry to tell you that the reports of fine-tuning's demise are greatly exaggerated. RAG and in-context learning are powerful, yes, but you're still paying per token for model input!

You're also beholden to high latencies with external services! You can fine-tune an 8B model for a specific purpose with much lower per-token latencies than using a larger general model.

Conclusion

So this is what I understand of the current state of the world from my very limited point of view as a web developer trying to figure out how to incorporate LLMs into an existing product.

I would love to learn more about Task Oriented Dialog systems and get a better sense of the environment in which they exist. Please reach out! I'd love to hear from you if you've worked on systems like this before.

Other Resources

Moving onto a new chapter

2023-06-03T00:00:00Z

Moving onto a new chapter

I left Noom on June 5th, 2023 after 5 years. I wanted to write about a few of the highlights of my time at Noom, why I'm leaving, and where I'll be going.

I never thought Noom would take off the way it did. When I joined the company in 2018, it was small and recovering from its last round of layoffs. The company hadn't found product-market fit yet. We were scrappy and fighting to survive. I joined two other engineers on the Growth team and we went head first into rapid experimentation. We found product-market fit in 2019 and little did I know the next few years would be some of the most intense periods of my working life.

A few highlights:

I had the opportunity to build and architect an experimentation platform for Noom.com.
I helped scale the growth team from 3 engineers to 30 at our largest in just 2 years.

A few reasons I'm moving on:

I've battled burnout several times, a lot of it was due to my role as an Engineering Manager.
The company has grown up and has reached a stage where I feel less effective.

Highlights

The Meristem Experimentation Platform

We built our own experimentation system instead of buying something off the shelf. I know, I know. We re-invented the wheel! It seemed like the right decision to make at the time. To understand the solution we came up with, its important to understand the context in which we made those decisions:

We ran experiments using feature flags.
We were a very small team with a lot of product managers per engineer. Our PM/Eng ratio typically hovered around 1.5. That's 1.5 Engineers per Product Manager! To put that into context, for example, Netflix has a ratio of about 12! That's 12 engineers per 1 PM!
PMs were responsible for creating a hypothesis for improving Noom.com conversion rates, creating the specification, and project managing the experiment, QAing and doing data analysis afterwards. They typically had very full workloads! Our PMs did everything!
Engineers were in charge of implementing experiments and working with PMs to right size the amount of work for an experiment based on the ROI of the feature. Engineers are also in charge of maintenance of noom.com
The team prioritized experiment velocity (to a fault). A base expectation was that each engineer would create and launch around one experiment per week (depending on the difficulty of implementation). This meant that there was not a lot of time for things like cleanups or investing into the infrastructure due to the pressure to run more experiments and lack of "slack" on the engineering side to devote to tech debt cleanups.

These constraints meant that we were trying to experiment very quickly and that all available engineering time was typically devoted to building and experimenting with new features. By using feature flags, we were incurring technical debt for every experiment we ran. Not only that, but feature flags typically interacted in difficult to predict ways and required expensive processes to get merged and tested for every potential experiment we wanted to run. This was a lot of overhead!

My teammates Sumin and Hubert came up with a novel idea: "What if we could use Git to manage our experiment variation branches"? We'd be able to throw out the code for failed experiments easily since roughly 9 out of every 10 experiments were considered failures. Then we'd only merge code into our codebase that we knew we'd wanted to keep. By not having to merge code into our main codebase for every experiment, we'd save ourselves a lot of effort reviewing the code, testing the code, maintaining the code, and the removing it when the experiment ultimately failed. We were optimizing for experiment velocity while heavily constrained by engineering bandwidth.

I was put in charge of the project and built out an initial version. Surprisingly, it worked but we hit scaling issues pretty quickly. We re-built the system with a different architecture and it ended up working really well for us. The system isn't without its flaws that we've had to work around. If I were to build another experimentation system today, I'd probably choose somthing else, but for our constraints it worked well.

Scaling the team and going into management

I also started to transition into engineering management. I went from a mid-level Individual Contributor (IC) to Tech Lead, to Tech Lead Manager and finally to Senior Tech Lead manager. As I progressed, my focus shifted from technology to people and teams and although my title was Tech Lead Manager, I was effectively an Engineering Manager by the end of my time at Noom.

I made a lot of mistakes as a manager and I often describe management as feeling like you have an infinite number of ways to fail and very few ways to succeed. It's a stressful and thankless job with very long iteration cycles. I still don't think of myself as a particularly good manager, and I'm often haunted by an incessent feeling that I was letting my teams down.

I'd say my biggest accomplishments as a manager at Noom were the people I'd help grow and mentor who would go on to succeed themselves. I don't look back at my time as a manager and remember the metrics we helped move, or the features we shipped, or congratulations from my bosses. Its the people that looked to me for advice, that I helped promote, that I helped get recognition that I got the most satisfaction from.

By and far my biggest gripe with management are the administrative parts like promotion packets, performance reviews, Performance Improvement Plans (PIPs), Objectives and Key Results (OKRs), etc. As Noom went from a small startup to a larger and more mature company, there was more and more priority placed on standardizing and calibrating across the company. I've heard these are typical growing pains that companies around Noom's size faced, but I can't confirm or deny this posit since Noom is the only hyper growth company I've worked for.

When Noom rolled out a new performance review system that tied performance to compensation, I struggled a lot with this change. The goal was to reward and recognize high performers, which sounds like a really great goal. You want to make sure that those teammates that are bringing a lot of value to the team are recognized and rewarded. In practice howwever, as a manager it feels really shitty to have to discern who is deserving of higher raises or bonuses based on indivdual performance when you're actually trying to optimize for your team performing well. How do you gauge whether somebody is actually performing at a level above and beyond their own? We fell back to our competency matrix, which had a few main areas but also left a lot of room for interpretation. Not every engineer fit into this template, so some did extraordinarily well in some areas and less well in others. Some also did work on areas not covered by the matrix (like documentation or organizing team events). We did raise these issues and they did eventually get addressed, but it was an extremely painful transition.

Another thing I noted during my time moving from an extremely small and fast paced team to a large team was that focus and incentives shifted from the organization to individuals. What I mean by this is that as we grew, us managers became much more aware of the need to promote people, which meant we were incentivized to ensure engineers were checking off certain boxes to be able to promote them. A stark contrast from when the team was small and scrappy, we just worked towards a pure goal of keeping the company alive. As we grew, this goal was de-emphasized and a lot more focus was placed on ensuring people were happy, growing, and productive. I don't necessarily think this is a bad thing, but it was certainly not something I was prepared for.

Finally, I look back now and I realize how difficult it is to learn to be a manager as a group of first time managers with little support. I wish we had more experienced engineering managers to help guide and teach us as we went, because when we finally got a manager with deep Engineering Management experience, the experience was a lot more pleasant. However it was too little, too late.

I'd love to try engineering management again in the future, now that I've learned what the job entails and have a network of other engineering managers to learn from. But maybe I'll try working as an IC for a while first.

Why I'm moving on

I burnt out several times during my time at Noom. I'm pretty self-motivated and when I'm invested, I go hard. This meant that I would find myself working really hard and burning out, or being heavily invested in something at work and burning out if I lose agency or am required to "disagree and commit". This cycle of burning out is expensive on my psyche, and over time it manifests in negative thinking. I noticed that I'm sliding into this cycle so the best way for me to reset is to move on to a new role, taking these lessons with me.

I've also done a lot of introspection these last few months and realize that as Noom grew, it needed more process in lieu of the higher bandwidth communication that was easy when the company was 50 people. As organizations grow, you need structure to ensure people are working towards the same goal, this unfortunately means that companies undergo growing pains where people like me that joined early on are required to change and adapt to the new environment. The skills that make an individual successful at a small company are very different from the skills necessary at a larger company.

The incentive structures are also very different. Typically at startups, you'd find a lot of intrinsically-motivated people, working towards a goal. This pool of intrinsically-motivated people is exceptionally small, and as a company grows it needs to add extrinsically-motivated people to fill the ranks. Extrinsically motivated people respond to a very different incentive structure that can be at odds with intrinsically-motivated people.

What I realize is that as Noom grew, I often felt at odds with the cultural shift the company underwent. I don't know if I'm just better suited to smaller companies, or was unable to adapt to the cultural shift at the company, but I often felt at odds with the leadership and processes that were put in place to support a more mature company. I don't think its good or bad, just is a consequence of hyper growth. You can be extremely diligent about preserving your "culture" but things will still change.

Next steps

I'm excited to announce that I'll be moving back to an Individual Contributor (IC) role at Clockwise. They have a really amazing piece of technology that helps people maximize their time and abstract away the really annoying task of managing your calendar. I'm really excited to be joining the Growth team and working on projects like Clockwise AI.

How I use Obsidian

2023-02-07T00:00:00Z

Introduction

Obsidian is all the rage right now in the productivity tool market, especially with software engineers. Similar to my note-taking app Collate Notes which I built in 2016 it utilizes a Markdown file format with a YAML "front matter" section on top (a-la Jekyll's doc format) to handle document metadata.

This local-file first approach means that your data is durable, since it's just markdown files in a folder structure. Even if Obsidian goes out of business, the data is still there and easily usable.

It also means that you need to develop your own way of using it. The tool does not dictate how you organize your data, you need to come up with your own system.

My System

Keeping it simple

There are a lot of systems out there like the PARA Method which have several folders to organize your data depending on the context of the information. PARA requires an Archive, Projects, Areas, and Resources folder and has rules on where to store specific kinds of information.

I tried it for a while and realized that more often than not I would spend time trying to figure out if something was an Area, or Resource, or Project. It's not worth the mental overhead. I want a super simple system that just stays out of my way.

Organizing Knowledge

I have two "areas" in my note-taking system. An "Inbox" where I keep anything active that I'm currently working on or need access to, which is the root of the vault. And a Knowledge folder inside that where I put everything when I'm done with it. That's it.

It looks like this:

!Knowledge/
<everything else>

The workflow looks like this:

I want to write something down.
I create a new file in the root of the folder and write it down.
If I want to group stuff together, I create a folder.
Once I'm no longer working on this thing, or I don't need it anymore. Move it to the Knowledge folder.

The Knowledge folder has subfolders, that help me organize information into smaller buckets. I have buckets for things like:

My Home
Business Ideas
Finance
Fitness
Health
Knowledge Management
Work
Software Engineering
Writing

The main purpose of the Knowledge folder is to keep information around if I need it, but just tuck it away out of sight, so I can focus on the important stuff.

Task Management

The other piece of my system is task management. I've tried all sorts of apps out there to manage tasks, and this one is the simplest and keeps tasks close to knowledge.

For example, if I'm working in a document where I'm ideating, collecting information from various sources, or writing thoughts, it is handy to be able to drop tasks into the same document.

I use the Tasks Plugin to aggregate my tasks from across my whole vault, and tags to add context to tasks.

A task looks like this (Emojis are used by the tasks plugin to add metadata).

- [ ] Take out the garbage 🔼 📅 2023-02-08 #personal

I have a Tasks note where I aggregate all my tasks using a query and ensure it's grouped by tags.

```tasks
(not done) OR (done date is today)
group by tags
sort by urgency
short mode
```

Conclusion

That's the system that's currently working for me. Everyone comes up with their own system based on their preferences, so do what feels right for you. This is how I'm currently managing things, and I'll use it as long as it sticks.

I'm your manager, now what?

2023-01-27T00:00:00Z

Introduction

After a recent reorganization at my company, I'm taking this opportunity to start fresh with how I approach my relationship with my direct reports. I've found that my instructions to my reports are often handed down piecemeal, expectations aren't set, or I've just forgotten things I should be doing. That is not a great experience as a direct report.

This document is really meant to keep me accountable as your manager. It's here, so you understand what I can do for you and what I expect from you. I'll refine this document as I go, and if there is anything in here you disagree with, please let me know!

My general philosophy towards management

My overall philosophy can be summed in my blog post: The Grug Brained Manager. Don't feel like you have to read it unless you want to get a deeper understanding of my management philosophy. In a nutshell:

No Surprises. I can't promise you won't ever be surprised, but I will try my hardest to make sure you have full information as soon as I can give it to you.
Give me the technical details. I love the technical details, even if it's not my area of expertise. I'm happy being a rubber duck or listening to you deep-dive into a technical issue. I may not be much help, but I'll always listen.
I trust you. By default, I will trust you to do your job to the best of your ability. I will also trust you to tell me if you need help, and I will do everything in my power to help you. Remember that trust is easy to break and difficult to repair, so I ask you to level with me whenever possible, and I will try to do the same.
Feedback is hard. I will do my best to give you feedback with empathy and kindness, and work with you if there is something you can work on.

Expectations

Onboarding

If you're new to the company, congrats! I'll be helping to support you during your onboarding period. You should be creating onboarding goals during your first few weeks on what you'd like to accomplish with my help.

I'll supply you with our onboarding materials, an onboarding buddy, and make myself available to answer questions.

I understand onboarding is a pretty daunting time. There is a lot of new things coming at you very quickly. I don't expect you to jump in coding right away and prefer you ramp up slowly instead of drinking from the firehose. Let me know if you're feeling overwhelmed at any point and I'll try to throttle things back for you.

One on One meetings (1:1s)

Regular 1:1s are important for us to stay in touch and to ensure there is an open and available space for us to talk. My preference is weekly or bi-weekly, but I'll defer to you on the cadence.

You own the agenda for the 1:1s. I'll bring topics, but I'm expecting you to bring topics to discuss as well, and your topics will usually be prioritized above my topics. It's a good idea to spend a few minutes before our 1:1 to jot down some notes in our shared 1:1 document. This shared document is a great place to drop non-urgent questions or thoughts between meetings so you'll populate the agenda as you progress through the week.

Some things we can discuss in your 1:1:

What are you working on? What problems are you encountering? Where are your bottlenecks?
- By feeding me information on things that aren't going well, I get a better sense of what's happening on the ground and where problems maybe forming. This is super valuable!
- Do you have an idea for a project? Let's talk about it.
- Did you encounter an issue or bug somewhere? Let's talk about it.
- Do you have any ideas on how to improve our workflow? I'd love to hear it.
Let's talk about your career goals and your next step.
- It's totally fine if you have no idea what your goals or next step should be, I can help guide you there with this three part career conversation.
- Is your next step a promotion? Let's talk about the timeline, how the company promotes, and what you will need to do to assemble a promo packet.
Let's talk about your performance and performance review
- How familiar are you with the company's performance review system? Let's talk about it.
- Do you have a brag doc? Let's take a look at it or update it! A brag doc will help me represent your work in the best light during performance reviews.
- Do you know and understand the company's expectations for you in your role at your current level? Is there any part of your role you don't understand?
Curious about how parts of the company work?
- I'll do my best to answer your questions or find the answer for you.
- Do you have any questions about your benefits or compensation? I can try to answer them or point you to somebody that can.
Curious about my take on a situation? Just ask.
Do you want to vent about something? Go ahead, it's a safe space and between us.
Tell me how you're feeling. Is everything going well outside work? Have you been distracted lately? Is there anything I can do to help (time off, too much work on your plate, etc.)?
Need advice? I can do my best to help guide you through whatever it is you're going through.
Are you happy in your role? If there is something you feel is missing, let me know so we can work on it.

For my direct reports that are also managers:

How is your team doing? Are you encountering any issues with your direct reports?
What is your team working on?
Do you have any situations with your reports where I could provide guidance?

You don't need to wait until our 1:1 though. Feel free to schedule time with me or ping me on slack at any time.

Brag Doc

Help me help you. I may or may not have a view of what you do on a day-to-day basis. When it comes time for promotions or performance reviews, you are your own best advocate. I will do what I can to advocate for you on your behalf, but you need to give me ammunition to make a strong case.

A brag doc or work log is a simple list of the things you work on so I can take that and transform that into a performance review or promotion packet.

By updating it at a regular interval, you don't need to do the "what did I work on a month ago or more" dance. Take a minute or two every week to just jot down what you've been working on and when performance review season comes around, you don't have to do anything!

Having a running work log/brag doc is also useful when it comes time to update your resume. You'll have a full accounting of your work that you can easily pull highlights from.

Promotions

Going for promotion is your choice. I will happily assist you in trying for a promotion if it's your goal, but I won't force you if you're happy where you're at.

Promotions take time to plan for and to build a case for. Managers are expected to assemble a "Promo Packet" to be submitted to be reviewed. This promo packet makes the case for you and your promotion. You'll typically need to prove you exhibit the skills of a person of the next level for a minimum of six months. We'll also need to collect endorsements from people you work closely with, which I'll take care of.

Let me know you want to try for promotion as soon as possible. It may take planning to build a case, and I'll guide you along the way.

Time off

I support you taking all your Paid Time Off (PTO) when you'd like to take it. Please use your time off and let me know how I can support you, so you can use it! I will almost always approve PTO and generally believe the team should support your time off and not vice versa.

In terms of logistics:

Please follow whatever PTO policy the company has laid out.
Please make sure your calendar is marked without of office (OOO).
A 2+ week notice before your PTO would be appreciated, but I understand if it's last minute, and we talk about it.
Make sure your team and any stakeholders know you're taking PTO, and remind them as it gets closer and closer.

Working hours, location, etc.

Personally, I don't care what hours you work, or where you work, as long as you:

Have your working hours set in your calendar.
Attend all meetings.
Have at least a 4-hour overlap between your hours and the general company working hours.
Are responsive and get your work done on time and to a high standard.

However, the company might have policies in place. Please follow the company policy first, and my personal policy second.

I generally encourage people to have set working hours in which they are actively working and available, and then "offline" the rest of the day.

Communication

With remote work, it's really important to be communicative. Since we don't have your physical presence in an office, being over-communicative in our team messaging software is important!

I will never expect a response from you if I message or email you off-hours. If it's an emergency, and you're needed, you will be paged.

Your work

I generally expect your work to be mostly bug free, conform to existing standards, and is of a high-enough quality commensurate to your level. I don't expect perfection, but I do expect improvement if you receive feedback.

The most important thing is showing enthusiasm for your work, working well with your team, and treating everybody with kindness and respect. Being good at your job doesn't mean being a brilliant jerk, it means being somebody people trust and respect to get the job done.

Conclusion

I hope this post was able to help form a basis for our working relationship as manager / direct report. I want to make sure you have the full information to ensure you're meeting my expectations, and I want to make sure my expectations are clearly spelled out without any ambiguity.

Check back from time to time as this doc might be updated as I continue to grow as a manager.

Assessing tasks as a product-minded engineer

2023-01-25T00:00:00Z

A good product-minded engineer is worth their weight in gold. One critical skill these engineers have is the ability to look at the business and understand why they are asking you to build what you're building. Then figure out if what is asked of you will achieve the intended goals.

Understanding the business

First, if you don't understand how your business makes money (or intends on making money), and how your role and team fit into that equation, then you need to stop whatever you're doing and figure this out. Businesses are in the... business... of making money. Everything else is window dressing.

Second, you need to understand the priorities of the business. What does the business want you to focus your efforts on to help the business achieve its goals (of ultimately making money)? Your team should have its own goals (which at a bare minimum you should know... right?). Understanding how your team's goals roll up into the company's goals means you have a clear path to... drum roll please... understanding how your work makes the business money.

Third, you need to understand the tempo of the business. Is it wartime or peacetime? Is it business as usual or is everything on fire? If your company has good leadership, they will flat out tell you. Otherwise, it's usually intuitive. However, you'd be surprised how many people are oblivious to their environment, only looking up to discover everything is on fire. Tempo dictates your work, and how you produce it. It can also be a proxy for how the company is doing. The big-brained move is to fully understand your company's financial position (as well as how to assess it) and use that to understand what the tempo should be.

Time and complexity tradeoffs

When approaching a task, there is usually an optimal solution and a hacky-as-hell-yuck-this-is-bad-don't-do-this-why-would-you-do-this-please-stop solution.

There are an infinite number of ways to accomplish a goal in software engineering. When approaching a task, there is usually an optimal solution and a hacky-as-hell-yuck-this-is-bad-don't-do-this-why-would-you-do-this-please-stop solution. A lot of software engineering literature out there focuses on the optimal solution, which in a vacuum, is all well and good.

However, the business is probably spending a lot of money on your time. There is an inherent return on investment (ROI) calculation done on your time versus the impact of implementing your solution. As engineers, we don't usually know how much the business is willing to spend on a particular task.

If you're more junior and have a Product Manager, it's easy to just present them a menu of options and have them pick. Give them the pros and cons of each solution and let them decide. If you have a particular solution you think is worth doing, make sure they know what it is and why.

More senior engineers, when equipped with a deep understanding of the business and tempo can make the call themselves on the optimal solution. When asked, they can provide the rationale of why they picked that particular solution. Product Managers usually trust these engineers to make the right call.

Experimentation reading list

2023-01-24T00:00:00Z

Introduction

During my time on the Growth team at Noom, I've had to learn a lot about online controlled experimentation. I've put together most of the public resources I've come across into this document.

If you're brand new to online controlled experimentation, I high recommend starting with Trustworthy Online Controlled Experiments : A Practical Guide to A/B Testing. Its a detailed guide on the experimentation landscape. It was a must-read for every member of my team, and was known as the "Experimentation Bible".

Whitepapers

Overlapping Experiment Infrastructure: More, Better, Faster Experimentation
Online Controlled Experiments at Large Scale
Controlled experiments on the web: survey and practical guide
Democratizing online controlled experiments at Booking.com
Seven Rules of Thumb for Web Site Experimenters
SQR: Balancing Speed, Quality and Risk in Online Experiments
Growth Book Stats Whitepaper
Success Stories from a Democratized Experimentation Platform
- Notes:
  - Netflix experimentation platform
  - Interesting: Netflix uses R/Python notebooks so people can contribute reports, statistical methodologies, and metrics.

Books

Trustworthy Online Controlled Experiments : A Practical Guide to A/B Testing

Videos

Online Controlled Experiments: Lessons from Running A/B/n Tests for 12 Years, Ron Kohavi 20151130

Statistics

Stanford Seminar: Peeking at A/B Tests - Why It Matters and What to Do About It
- Notes:
  - Frequentest
    - Popular 100 years ago
      - Data was expensive to gather back then.
    - Fixed sample size testing
  - Peeking problem
    - Continuously monitor results dashboard
    - Adjust test length in real time (Adaptive sample size testing)
    - Peeking can dramatically inflate false positives.
    - Intuition
      - Sample size specifies a point in time, doesn't say anything about what happens between the start of the experiment and when you hit sample size.
      - High chance of significant result on the way to sample size may bias whether we keep the test running or not.
      - If you wait long enough, there is a high chance of an eventually inconclusive result looking "significant" along the way!

Experimentation Platform

Experimentation Platform - Microsoft Research
Building our Centralized Experimental Platform | Stitch Fix Technology - Multithreaded
- Notes:
  - They have analytics automated
  - Not a lot of details in this blog post
Meet Wasabi, an Open Source A/B Testing Platform
- Wasabi not under active development
Scaling Airbnb's Experimentation Platform
- Notes:
  - Uses Airflow for meteics
    - ERF runtime from 24 hours to 45 minutes
  - Useful article to learn about how they do data processing
It's All A/Bout Testing
- Notes:
  - Targeting and allocation
    - There is one more topic to address before we dive further into details, and that is how members get allocated to a given test. We support two primary forms of allocation: batch and real-time.
    - Batch allocations give analysts the ultimate flexibility, allowing them to populate tests using custom queries as simple or complex as required. These queries resolve to a fixed and known set of members which are then added to the test. The main cons of this approach are that it lacks the ability to allocate brand new customers and cannot allocate based on real-time user behavior. And while the number of members allocated is known, one cannot guarantee that all allocated members will experience the test (e.g. if we’re testing a new feature on an iPhone, we cannot be certain that each allocated member will access Netflix on an iPhone while the test is active).
    - Real-Time allocations provide analysts with the ability to configure rules which are evaluated as the user interacts with Netflix. Eligible users are allocated to the test in real-time if they meet the criteria specified in the rules and are not currently in a conflicting test. As a result, this approach tackles the weaknesses inherent with the batch approach. The primary downside to real-time allocation, however, is that the calling app incurs additional latencies waiting for allocation results. Fortunately we can often run this call in parallel while the app is waiting on other information. A secondary issue with real-time allocation is that it is difficult to estimate how long it will take for the desired number of members to get allocated to a test, information which analysts need in order to determine how soon they can evaluate the results of a test.
Twitter experimentation: technical overview
- Notes:
  - Uses feature flags
  - Pretty unremarkable
Zalando Engineering Blog - Experimentation Platform at Zalando: Part 1 - Evolution
- Notes:
  - Uses craw/walk/run phrasing a-la Ronnie K
  - Their system allows for staged rollouts
Under the Hood of Uber's Experimentation Platform
Notes:
- Good details, screenshots of their XP.
  
  Broadly, we use four types of statistical methodologies: fixed horizon A/B/N tests (t-test, chi-squared, and rank-sum tests), sequential probability ratio tests (SPRT), causal inference tests (synthetic control and diff-in-diff tests), and continuous A/B/N tests using bandit algorithms (Thompson sampling, upper confidence bounds, and Bayesian optimization with contextual multi-armed-bandit tests, to name a few). We also apply block bootstrap and delta methods to estimate standard errors, as well as regression-based methods to measure bias correction when calculating the probability of type I and type II errors in our statistical analyses.
- The XP detects major issues during analysis:
  - Sample size imbalance (Sample size ratio in control and treatment group)
  - Flickers (Users that have switched between control and treatment groups)
- They do data pre-processing:
  - Outlier detection using a clustering algorithm to detect and remove outliers
  - Variance reduction using CUPED Method
  - Remove pre-experiment bias using difference in differences to correct pre-experiment bias between groups.
- Sequential testing
  
  This is very similar to the circuit breaker system we were interested in building.
  
  One use case where a sequential test comes in handy for our team is when identifying outages caused by the experiments running on our platform. We cannot wait until a traditional A/B test collects sufficient sample sizes to determine the cause of an outage; we want to make sure experiments are not introducing key degradations of business metrics as soon as possible, in this case, during the experimentation period. Therefore, we built a monitoring system powered by a sequential testing algorithm to adjust the confidence intervals accordingly without inflating Type-I error.
  
  Using our XP, we conduct periodic comparisons about these business metrics, such as app crash rates and trip frequency rates, between treatment and control groups for ongoing experiments. Experiments continue if there are no significant degradations, otherwise they will be given an alert or even paused. The workflow for this monitoring system is shown in Figure 6, below:
- Bandit aka Continuous Experiments
  - Bandit is used at Uber for:
    - Content optimization
    - Spend optimization
    - Hyper-parameter tuning for learning models.
    - Automated Rollouts
Building an Intelligent Experimentation Platform with Uber Engineering
- Notes:
  - Uber uses their XP to do staged rollouts.
  - On metric computation
  The new analysis tool does not pre-compute the data of the metrics, which cut down on our data storage spend and reduced our analysis generation time. Now, when the data is ready for analysis, we use a SQL query file to generate metrics on the fly whenever people make a request on the WebUI. After that, we use Scala as our service engine to compute the probability (p-value) that the treatment group mean is significantly different than the control group mean, determining if the experiment reached the target sample size.
In-house experimentation platforms
- Cool list of various companies experimentation platforms.
ExPlat: Automattic's Experimentation Platform
- Notes:
  - System is called Abacus
  - Open source

Easy dynamic social sharing image with eleventy

2023-01-23T00:00:00Z

Introduction

I'm in the process of rebuilding this website using eleventy and I wanted to get fancy with this revamped site.

One feature I wanted was a custom Open Graph social share image for each page on the site. If you're not familiar with Open Graph, check out this explanation. In a nutshell, when you share links to the site on social networks, it should an image to draw people into the page.

Eleventy has a pretty robust ecosystem already and lo and behold, there are already plugins to handle this for you. Great, let's just check them out.

What's out there?

eleventy-plugin-social-img pre-generates cards using a shortcode. I spent about 30 minutes trying to grok the readme and how to use the plugin. I like that you could create an HTML template, but the plugin looked overly complicated for what I needed.

@resoc/eleventy-plugin-social-image only works with Netlify and requires a server component. This site is hosted on GitHub Pages and is fully static. Pass.

eleventy-plugin-social-share-card-generator requires a Cloudinary account. Pass.

eleventy-plugin-og-image looks promising, pre-generates images, and looks pretty easy to use. This could be a runner-up if I ever want to go a different route.

All in all, I wasn't a huge fan of these solutions. There had to be something simpler.

Something simpler

The eleventy team offers various API Services for the public to use. Screenshots is a runtime service that takes a screenshot of a URL and returns that image (and caches it). Bingo!

All I need to do is create a simple template and style it however I'd like for my social sharing card. Apply the template to every page in the site. Then the service can generate screenshots for my blog posts on demand when somebody shares a link to my site.

Creating the card template

To make it easy to manipulate URLs, I would have a mirrored set of paths at https://patricklee.nyc/social/<page URL>. Then I simply embed that URL to the screenshot service API URL and it would return an image.

Example of the social card webpage:

https://patricklee.nyc/social/about

Example of the social card screenshot image generated by the service:

https://v1.screenshot.11ty.dev/https%3A%2F%2Fpatricklee.nyc%2Fsocial%2Fabout%2F/opengraph/_1674507886315

Note: You need to URLEncodeURI the page being passed into the API

I created the simplest webpage possible, you can see the template here.

I wanted a simple design with the title of the page, the URL of this site, and a picture of me.
I created a 1200 x 630 pixel box, then added my card contents in there. The screenshot service accepts a size parameter opengraph which will take a 1200 x 630 pixel screenshot.

Generating the pages

Next, we need to generate the pages on the build. To do this. I created a page social.md with some basic frontmatter. See the page here.

---
layout: social.liquid
pagination:
  data: collections.all
  size: 1
  alias: post
permalink: "social/{{ post.url }}/"
---

This page uses a combination of collections.all, pagination, and permalink to force eleventy to generate a page for every page on the site under the /social/{{ post.url }} namespace.

Now when we want the social image for any page we simply add /social/ to the URL.

For example, the page https://patricklee.nyc/about has a social image at https://patricklee.nyc/social/about.

Updating the Open Graph tags

The last part is to update the Open Graph tags to use this image. In my header, I had my Open Graph social tags available. I just needed to create a URL that would take each page's URL, and append /social to it, then embed it into the screenshot service's URL scheme.

To make things dead simple, I just created a shortcode that would generate this URL when used on a page.

// eleventy.js
eleventyConfig.addShortcode("openGraphScreenshotURL", function () {
  // URL Encode the page
  const encodedURL = encodeURIComponent(
    `https://patricklee.nyc/social${this.page.url}`
  );
  // Generate a cache-busting key for quicker testing
  const cacheKey = `_${new Date().valueOf()}`;
  // Return the screenshot service's URL to add to the open graph tags.
  return `https://v1.screenshot.11ty.dev/${encodedURL}/opengraph/${cacheKey}`;
});

Then I updated my header to use this shortcode under the og:image and twitter:image tags like so.

<!-- header.liquid -->
<meta property="og:image" content="{% openGraphScreenshotURL %}" />
<meta property="twitter:image" content="{% openGraphScreenshotURL %}" />

That's all it takes!

Conclusion

The risk here is if the eleventy team decides to sunset the screenshot API, I'll need to switch to a solution that pre-generates the social share images or host the API myself. Considering how little traffic this site gets, I'll take that risk for reduced complexity now.

I like this solution ~~because I came up with it~~ because it is simpler, doesn't require pre-generating all the images upfront, and doesn't require adding more packages. Oh and because I was able to offload a lot of the work to the wonderful devs at eleventy. Thanks, ya'll!

The Grug Brained Manager

2022-08-23T00:00:00Z

The Grug Brained Manager

this collection of thoughts on managing software development teams by a grug brained manager based on grug brained developer.

grug brained manager not smart. somehow managed to convince boss that grug capable of managing others. grug barely capable to polish mouth stones.

grug pretend grug know what do. grug talk to other grug managers and realize nobody know what to do. grug realize management not as simple as development.

over time grug learn painful lesson after painful lesson and decide to write on stone tablet so young grug manager not make same mistake.

Managing People Is Hard

nobody told grug managing so hard. grug used to be okay grug brained developer.

wanted become grug manager for more shiny rock and to be grug leader. grug no realize managing not same as developing. grug very happy developing, feel good when make shiny object and solve problem.

grug no realize in management, there no shiny object to hold up proudly. type of problems grug solve take long time to see results. grug has lots of self doubt when think what to do, but grug need to make decision and commit.

there not a lot of support as manager. grug find that there not a lot of process or materials for grug to learn to do job. when grug was developer, lots other grug make content to learn job. as manager, not as much information for grug on how to do actual job.

grug introvert, that why grug get into making software. management however is purely people-focused, which means job is more taxing for grug. grug realize that in management, easier for grug to burn out. grug more tired at end of day. grug need to make sure to make time for self care, but hard to balance. grug burned out several times and sometimes just want to take a six month sabbatical to recover.

grug not prepared to deal with unhappy people. grug no like conflict, however grug must deal with conflict in job. when grug responsible for unhappy person, it weigh heavily on grug mind and grug want to make situation better. this add extra stress on grug.

grug realize that management takes much larger emotional toll on grug than developing. grug must work extra hard to not show emotion because team looks to grug as leader. sometimes grug feel strong emotion in pit of stomach but must force it to go away. this is hard for grug.

it not all bad though.

grug get satisfaction from seeing young grug learn and get promoted. when grug team get shiny rock and praise, grug feel good for them. grug no do this for glory, only shiny rock and see grug team solve problem and be happy.

grug also get satisfaction from helping other grug do work. grug get sense of satisfaction from stopping grug product manager from other team bugging team member. grug chase them with stick and beat chest.

grug also get more shiny rock for constant headaches.

grug was okay grug brained developer, but there many more smarter grug brained developer. senior grug developer position rare and there are many more smart and passionate developer that will overpower grug.

grug not want to develop software in free time, grug want to live life with family and provide many shiny rock for them. grug took realistic look at grug career path and decide that grug can progress in management career more than developer career.

grug has skill other developer may not have. grug good at writing on stone tablet. grug enjoy think about how team can work better. grug enjoy helping lead team so company get more shiny rock and help customers. lots of grug developer only want to concentrate on making shiny object shinier, grug want to make sure customer enjoy shiny object.

grug want young grug to think hard when want to try management. management not the right career path for all grug, but hard to know unless young grug try it.

Rule #1: No Surprises

grug learn over time that number one rule is no surprises. surprises bad. surprises very bad if it affects shiny rocks grug’s team takes home. breaking this rule can lead to very bad consequences like difficult conversations or a club over grug’s head.

grug realize biggest reason for surprise is grug expect one thing and direct report expect another. communication the most important thing for grug manager to do right.

grug believe grug need to make expectations explicit up front with grug report. grug need to make sure direct report know what grug expect of them. direct report need to know what and how to do their job, if they do not, grug fail as manager.

grug also believe that direct report should make expectations explicit with grug. grug need to know what direct report want and work with report to help them get it. grug fail as manager if he no know that.

grug learn hard way that sometimes grug avoid hard conversation because it make grug uncomfortable. grug make excuse for self or ignore problem. then small problem become big problem. grug embarrassed this happen many times before grug learn. it still happen but grug used to embarrassment now.

other time grug not know what expectations should be for grug developer. company leadership failed grug by not teaching grug how to be manager, but that not good excuse. grug learn many painful leadership due to grug’s lack of experience and support.

Feedback

Getting feedback for reports

grug no have eyes everywhere. grug has limited time and too many meetings. it sometimes difficult to know how direct reports are performing. even when grug in same team as direct report, sometimes it difficult for grug to know what direct report doing.

grug learn that getting feedback is hard.

grug learn that grug need to meet with people that work with direct report and build relationship with them. grug needs them to trust grug so they will open up and give grug honest feedback.

many people uncomfortable with giving honest feedback, especially to coworker’s manager. grug need to convince them that grug intentions good and that grug want to help direct report become better teammate.

grug learn hard lesson when grug not collect feedback for direct reports. when direct report under performing, grug not know because nobody told grug. only when situation got very bad did grug know and by then grug had failed. grug learn that grug need to go and get feedback from people.

Giving feedback to reports

grug learn that giving feedback is hard.

grug learn that giving feedback is one of the hardest part of management. grug not used to telling people bad things about them to their face. in real world, if grug give random person on street feedback, they hit grug with club.

in business, grug learn that giving feedback can mean the difference between a good and bad team. luckily most of the time grug no get hit with club.

grug need to mix own feedback for direct report and feedback from report’s team. grug learn that it takes work to figure out where direct report can improve. grug then needs to give direct report feedback.

when collecting feedback for direct report, having concrete examples for negative feedback helps people connect their behavior to real situation. when people come to grug with feedback for direct report, grug try to ask for specific situation. when grug see behavior grug want to give feedback on, grug try to write it down.

grug learn that giving negative feedback between positive feedback very bad. smart managers call it “shit sandwich”. grug no like shit sandwich.

grug realize at end of the day, giving feedback to report like ripping off band aid. grug need to do it and get through pain. key thing grug learn is that grug always try to have good intention for direct report and that benefit should outweigh the cost.

over time, after grug do it more, giving feedback not as scary, but still hard.

Managers should stay technical

grug firmly believe that managers that are close to developers should make time to work with the technology.

grug realizes that the higher you are on the food chain, the more valuable your time is. there are job you can not delegate so those job should take precedence over job you can delegate.

grug has worked with middle managers that are not technical. grug does not know how they make wise decisions. non-technical managers must rely on their team to make decisions. grug does not want to sit in the middle and relay messages. non-technical managers also lack ability to understand if things their team talk about make sense.

grug consults technical members of team and defers to their decision. but grug is able to make better decisions than the non-technical managers.

grug mostly spend available time working on small problems that help the team’s productivity. grug tries to stay out of the way as much as possible. grug fix bug here, implement nice to have feature there.

grug believes that in order to lead people, grug needs to stay grounded to what life is like for them.

Hiring & Firing

Hiring

when times are good, grug get approval to hire more grug developer for team.

grug learn that hiring grug on to team is risky. new grug developers on team cost the team productivity for several months while they ramp up. new grug developers can also change the team dynamics. grug needs to be careful when bringing in new developers.

grug learned that sometimes there is a mismatch between the hiring process and the day to day work. grug disagrees with difficult tests given in the hiring process, but grug doesn’t know how to do any better. grug also learned that the candidates that were going through the hiring process are very talented. grug sometimes wonders if the company could do better, but does not know how to fix it.

grug learned that interviews go both ways. grug needs to be a good representative of the company and try to sell the company to the candidate. grug realizes in software development, talented candidates have many options, and grug wants to only hire talented candidates.

Firing

grug believes that firing somebody should be a last option and if it gets to that point, then grug has failed. grug does not want to negatively affect somebody’s life until grug has exhausted all other options.

grug believes that its important to have empathy when dealing with these hard situations.

grug believes that identifying potential problems early and having difficult conversations early can help avoid issues.

however, sometimes there is a communication issue or misaligned expectations. grug tries to be very explicit with the report on what exactly grug needs to see the report improve on.

sometimes you give people many chances and they still don’t meet grug’s expectations, so grug then needs to make the hard decision to fire them.

Processes

grug once hear phrase “process is just expectations made explicit” and grug very much like this.

grug believe that like software, it is easy to prematurely optimize process within teams. grug tries to not have process until its necessary.

process can slow people down. grug hear from other grug often that they need to deal with “red tape”. process for the sake of process can get in the way of grug’s team doing their job.

process should be challenged from time to time. grug thinks that just because a process exists, doesn’t make it law. processes need to be re-evaluated to make sure they are still beneficial.

Keeping it simple

grug learn long ago during software development days that complexity is the external enemy. grug internalized this and tries to reduce complexity as much as possible.

grug tries to teach their team the ways of the grug brained developer as much as possible.

grug firmly believe that the right amount of code is 0 lines. grug wants to judiciously delete code whenever possible and make sure existing code is simple.

when making technical decisions, grug considers who will be working on it. code should be accessible to even the most junior developer. writing code should be simple and obvious.

Leadership

grug had vision of leadership from movies where heroic leader charges enemy lines from the front. the reality of leadership though is that grug goes around making sure team is not blocked by anything, they like what they are working on and are happy, and that the team has what they need to do their jobs.

grug’s main job is to make sure the team functions well and that ultimately they hit their goals.

grug believe strongly that leading by example is important, so grug tries to live up to expectations grug has for team as much as possible. grug hear once that a leader shouldn’t ask their team to do something they wouldn’t do themselves and grug nod head vigorously.

grug also believes that decisions should be made with empathy, that there is an inherent power imbalance between manager and worker that should be kept in mind.

at the end of the day, grug just try to do good job as best as grug can. grug realize that there no true ideal for what a leader is. grug just try to do what is best for team members and company and try to hold together as much of the team as possible.

Communication

communication number one thing to get right as manager.

clarity of communication is important skill for manager to have. grug managers can express complex ideas in simple terms.

grug try not to assume people know thing and will check or give simple fast explanation for thing people may not know of. grug often see people afraid to ask clarifying questions and tries to make communication accessible.

grug often need to synthesize information from various sources and combine it into coherent thesis. it important that grug listen to people and ask questions to understand underlying problem.

written communication is very important for remote team. grug has team members across many time zone, written asynchronous communication ensures people can engage when it works for them.

when grug must do a presentation, grug believe strongly in Guy Kawasaki’s 10/20/30 presentation method.

grug know that when manager suggest thing, it easy for people to take it as mandate. grug try to ask questions when possible to guide thought process instead of suggesting thing. grug also try to set expectation that grug not close to problem and that grug defer to team as technical experts.

grug try to use simple language whenever possible. grug has many English as second language team members and wants to make sure they can understand grug.

Grug like

Conclusion

grug think that key to management is to communicate expectations clearly and set up team to meet those expectations.

Resurrecting RSS feeds

2022-06-15T00:00:00Z

The good ole days

Update (2023-01-21): I've migrated this blog to eleventy and use its built in RSS/Atom feed generator now.

Back when I was a young lad, RSS feeds were the way to stay up to date on news, blogs, and other content. You'd find sites you liked, then add their content to your RSS feed reader and then you'd be able to go to a single place to view all your news and content. You'd be able to curate a large number of sites and read new items on a daily basis. Those were the days!

Then digital advertising got big, there was real money to be made from eyeballs on your website, so slowly sites began guarding their RSS feed content, ultimately getting rid of them altogether. You wanted people engaging with your site, not with your content, and slowly content began shifting to social media.

Back then, all you needed was your Google Reader feed. These days, content is gated within small walled gardens. If you aren't a member of these walled gardens, forget seeing the content! When my feeds started dying, I switched to Digg and then Reddit and Hacker News.

The thing I miss most about syndicated feeds, is that it was so much easier to receive updates for smaller bloggers and news sources. Social media and Reddit are great at surfacing new content, but if I want to follow along with a small time writer, there isn't much recourse. These days you'd have to follow somebody's Twitter or their Substack, but it just isn't as easy as it used to be.

Today I set up a RSS reader (NetNewsWire)and I'm going to attempt to curate my feed with authors I'm interested in hearing from. I also added a RSS feed to this blog, so others can subscribe to my feed and read what I'm writing about. It won't replace Reddit or Hacker news for me, but I hope I can supplement that content!

Subscribe to this blog's feed.

Aside: Adding a RSS feed to this blog

Since I decided to go ahead and re-invent the wheel for this personal blog and build my own Pandoc powered static site generator, I needed a way to generate RSS feeds.

The RSS 2.0 spec is pretty simple, so why not just generate my own rss.xml file then?

Using a pretty simple bash script, I just parse my directory of markdown files and assemble the feed:

function generate_RSS_feed {
    echo "Generating RSS feed"
    declare items
    while read -r file; do
        url_path=$(echo "$file" | sed "s:docs\/::g" | sed "s:\.md::g")
        date=$(grep "created:" "$file" | sed "s/created: //g")
        converted_date=$(convert_date_format_822 "%Y-%m-%d" "$date")
        title=$(grep "title:" "$file" | sed "s/title: //g")
        html=$(pandoc "$file")
        items+=$(cat << END
    <item>
      <title>${title}</title>
      <description><![CDATA[${html}]]></description>
      <link>https://patricklee.nyc/$url_path</link>
      <pubDate>${converted_date}</pubDate>
    </item>
END
)
    done < <(find "docs/blog" -name '*.md' ! -name "index.md")
    pub_date=$(date -R)
    template=$(cat << END
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">
  <channel>
    <title>Patricklee.nyc</title>
    <link>https://patricklee.nyc/</link>
    <description>Software engineering and general geekery</description>
    <language>en-us</language>
    <pubDate>${pub_date}</pubDate>
    ${items}
  </channel>
</rss>
END
)
    touch "build/rss.xml"
    echo -e "$template" > "build/rss.xml"
}

All this does is parse the frontmatter section of the markdown file to get some basic metadata, then use Pandoc to convert the file to html. It then assembles the XML file and writes it to rss.xml. This function runs at the end of my blog's build chain, but it can be run isolated as well.

See the script here.

I'm actually quite happy with this collection of bash scripts and Pandoc to generate my static site. The whole thing is stupid simple, there is zero magic, its ugly but functional and it is very narrowly scoped. There is very little that can go wrong and not many dependencies. For a blog that I want to infrequently update and keep around for a long time, its perfect and won't rot once a new JS framework gains popularity.

Note taking hell

2022-05-20T00:00:00Z

Note taking hell

Update(Feb 7, 2023): I've written down how I'm currently using Obsidian

I have a hard time sticking with a note taking system. I think at this point I've tested all the major note taking tools:

Evernote
OneNote
Simple note
Turtl
Google keep
Standard Notes
Workflowy
Bear
todo.txt
plain text markdown files
Omni notes
Apple Notes
Obsidian
Notion
Roam Research
Athens Resarch
Coda
Mem.ai (currently trying out)

I've also built and sold a now defunct note taking app called Collate Notes as well as a VS Code plugin called VSNotes.

I can't settle on anything! My notes, tasks, etc are scattered across various services and no one service seems to be able to keep my attention for long. I keep looking for that note taking panacea that will organize my life and help me become more productive, however everything I try always seems to come up short in some way, shape or form.

Notion is a favorite of mine, it had really great flexibility... but too much flexibility. It functioned great for writing nice looking documents, but failed at jotting down small tidbits quickly. I felt like I spent too much time making documents look nice instead of actually using the tool.

Roam Research was really nice. It introduced me to Zettelkasten which is a neat concept. However I dropped it because Roam was expensive and lacked polish. In it's stead I tried Athens Research and Obsidian, both were pretty meh.

Plain text note taking is all the rage with minimalists. I'm a big fan of markdown for documentation (this blog is written in markdown). For notes, it was okay. Interoperability between services usually sucks. Pasting raw markdown into a Google doc will just result in a bunch of markdown formatting in a google doc, so that doesn't work. Syncing data is also a bigger pain with plain text. Do you use Git? A service like dropbox? Then on mobile your tool needs to work with these services.

It's a nightmare!

I have no idea what I want

Do I want a writing tool?
Do I want a way to manage tasks?
Do I want a way to be reminded of things?
Do I want a way to manage a knowledge base?
Do I need a bookmarking tool?
Will I ever look at these notes ever again?
How do I reconcile the Google Docs heavy world that my work org operates in?
Do I want a dashboard that I can keep up all day to jot down notes, see my meetings, etc?

I would love having all these things in a single tool, but that's not feasible. Its basically an operating system at that point. The services that try end up sucking. The services that don't end up lacking. Damned if you do, damned if you don't.

I'm so lazy

I would love to be able to just automate stuff too. Click a button in slack and a todo item pops up in my tool or bookmark this link and it shows up in my notes. Integrations are great, interoperability between services means that I can pipe data from this service into my online brain. However very rarely does it do what you actually want. APIs don't play nice, data isn't available, hell a lot of these services don't even have APIs. What is one to do?

The promise of a 2nd brain, in the cloud, ready to be queried that offloads the work of thinking. Isn't that the dream? To be able to think a thought and have it appear in a personal knowledge base to be ingested and understood by a machine? Then spat back out to you at the appropriate time and place? Barring having a hyper intelligent machine, what is one to do?

I think I need to designate scopes

I need to pick several tools and keep them in their lanes:

Calendar: Google Calendar
Reminders: Apple Reminders
Jotting down ideas: Whatever is nearby?
Writing: Google Docs / Markdown (depends on context)
Bookmarking: Raindrop.io

I think leveraging a tool like raindrop.io will help me keep all these various notes organized in a central location... at least until I get bored of maintaining it...

Update(Feb 7, 2023): I've written down how I'm currently using Obsidian

Striving for dispassionateness

2022-03-21T00:00:00Z

Striving for dispassionateness

Not influenced by strong feelings or emotions; impartial

Definition

As I continue on my leadership journey managing the Growth engineering team at Noom, one aspect of my personality I've been trying to work on is detachment. I have a mantra "Strive for dispassionateness" that I plastered on top of my personal goals to remind me to continue to detach myself from emotions. Dispassionateness or detachment as I've defined it is the process of removing emotion from the decision making process to reduce the impact of bias and to try to make the best decisions possible.

My coworker Nikhil, the VP of Growth at Noom is my model for dispassionateness. I've worked with him for four years now and in this time I've had a glimpse into how he handles making difficult decisions and accepting hard feedback. He does it with thoughtfulness, without ego, and kindness. Its quite impressive to see.

I'm an emotional person, I feel strong emotions which can bias my decision making process. This personality trait has hurt me in the past. I've been in situations where I have a hard time forming sentences, breathing gets difficult, vision blurs. Getting overwhelmed by emotions is truly an awful experience. The adrenaline pumps as the fight or flight response kicks in, paralyzing your mind in indecision, and impairing your ability to think. I'm a conflict-adverse introvert. I hate conflict, but as a people manager and leader there is no avoiding conflict sometimes. You will eventually get pulled into it, especially in a fast moving organization with many different people with their own biases, goals, and opinions.

Disconnecting yourself from emotions isn't burying your emotions. Its a process of introspection and learning what your current emotional state is, and either working to return to my baseline state, or removing myself from the situation. A few things that I've learned:

Understanding what affects my mood

If I'm hungry or if I didn't get enough sleep, I'll often become irritable.
When I exercise regularly in the mornings, I set myself up to be in a better head space.
I burn out if I don't take vacations often or there are other compounding stressors. How close to burnout is important for me to keep tabs on. The closer I am, the more erratic my behavior will be and the less passionate I will be about work.
As it gets closer to the end of the day, I get tired and can more easily get irritable.

Understanding when to stop and take a breather

Sometimes people send me a message or email that spike my blood pressure. I've found that if something triggers me, taking a step back to think about it or let it bake helps a lot. I've found a lot of success putting something down, then picking it up again the next morning when I'm fresh and in a better head space. The scope of problems tend to be smaller the farther you get from it and initial reactions are usually overblown.

Understanding when to dig in

Speaking of initial reactions, one of my flaws is overreacting based on my initial reaction. Its easy to do so, but generally taking some time to investigate and dig in before gauging the severity of it is beneficial and something I actively work on. Its very easy to experience something, scream "WHAT?", go on a tear and make things works. It takes discipline and detachment to stop, investigate, and ask questions, then thoughtfully determine the impact and next steps.

Understanding when to let things fall by the wayside.

If you're doing a good job, people are going to want you to do more. Knowing when to let things fall by the wayside (which is different from letting things slip through the cracks) is an invaluable skill.

Goodbye iterm2

2022-01-07T00:00:00Z

I've used iTerm 2 for a few years now but came to the realization that I don't actually use any of its features. Mac's built in terminal is more than enough for my needs. See ya later iTerm 2 and thanks for being rock solid all these years.

An ode to my linter

2021-12-30T00:00:00Z

Oh my dearest linter,
adding squiggly lines under my mistake.
Gently teaching me why I'm wrong.
How do I fix it?
Some help would be nice.
You autocorrected it! Thank you twice!

The documentation is right at hand:
"you idiot, don't add an ampersand!"
But I think you might be wrong,
my dear linter this is an edge case!
Well then I'll just have to add
a little comment to disable that
tiny check on line 35.
I'm sorry my linter,
you'll have to abide.

Dotfiles

2021-12-29T00:00:00Z

Finally got my dotfiles set up. Now I can have a consistent environment across both Mac and Linux. Really only using it for managing initial environment setup scripts and to manage my terminal environment. Will continue to evolve it as I need to but glad to have built out the basic scaffolding.

My walking desk setup

2021-12-28T00:00:00Z

Over the course of 2020 and 2021 I've become increasingly sedentary with lockdown and full remote work. Having gone from commuting into the office 1-2 times a week down to zero meant I had less opportunity to walk. Along with the birth of my son and having less time and energy to exercise I needed to find a way to integrate more walking into my routine. If I could figure out how to exercise while working then I would have more time to spend on more interesting things, a win-win situation!

I've seen treadmill desks before and was motivated after seeing one of my coworkers on one during a meeting. So I decided to pick up a treadmill and give it a shot.

This is my humble treadmill desk.

The treadmill

There are a lot of treadmills on the market of varying quality. I knew that I didn't need a high end treadmill since I was only going to walk on it. Space is an issue, so I wanted something that could fold away if I had company and needed to use my office as a guest room. Finally I wanted to make sure the product had a warranty and a presence in the US in case something happened with the product. There are plenty of dodgy looking walking treadmills on Amazon and I just didn't want to take the chance of getting a piece of junk.

This pretty much narrowed my choices down to:

While the LifeSpan TR1200 was a more solid choice, I decided to go with the smaller, sleeker and most importantly cheaper Treadly 2. My impressions of the treadly so far after about a month of usage:

Its heavier than I had expected. The build quality is good, it feels rugged and pretty solid.
I didn't get the one with the bluetooth and I kinda wish I spent the extra money and gotten it. Since I'm stationary at a desk, my Apple watch doesn't record the steps properly and being able to sync Apple Health to the Treadly would have been nice. It's no big deal though. You can upgrade to the pro version but it requires paying the difference and shipping back the existing treadmill.
A pile of black rubber accumulates at the back of the treadmill after a walk. I think it's rubber off the bottom of my shoes getting rubbed off, at least I hope it is. It would not be great if this was the belt disintegrating, but I don't see any wear on it so far.
The hand rail is covered by my desk so it's functionally useless, I just use the included remote to start and stop the treadmill. The handrail feels pretty solid though and it's folding mechanism is pretty solid.
It has a bluetooth speaker but I don't have a need for it.
The speed increments are in MPH and I've found my comfortable walking speed is about 2.1 MPH. I don't know how accurate speed or step counter are but that doesn't really matter all that much.
I only use the manual mode since I want the treadmill to set the pace and all I have to do is keep up. The Treadly 2 has an automatic mode that adjusts the speed based on where on the treadmill you're walking. Further up and it speeds up, further back and it slows down.

Overall I'm pretty happy with this treadmill!

The setup

I have a standing desk which I had planned on using as my treadmill desk. The plan was to set up the treadmill in the morning then move it out of the way when I was finished so I could sit/stand the rest of the day. I tried this a few days and realized that I needed to have a dedicated setup for the treadmill desk and that it had to live somewhere semi-permanently. Having to set up the treadmill attributed some friction to actually using it which meant I was less likely to use it.

So I cleared some space in my office, purchased a relatively cheap Tangkula adjustable standing desk off of Amazon and repurposed some extra peripherals I had laying around. I also purchased a Logitech C920x webcam for this setup so I could take video calls while on the treadmill desk.

On the desk: Split Keyboard, Apple Trackpad, wrist rests, Logitech C920x, LG 34" Ultrawide monitor, Ergotron monitor arm, Treadly 2 remote

When everything came, I realized that the standing desk needed to raised over the treadmill so it could slide underneath the desk setup. This is important because this cheap standing desk is relatively shallow and I needed to be centered in the middle of the treadmill so the desk had to be slid back. To solve this problem, I just bought some cheap 6" bed risers which gave the desk enough height so it could clear the treadmill.

I also have a small fan that blows me while I'm walking since I get pretty warm after walking a while. The fan helps a lot.

The desk is raised over the treadmill with the bed risers and the fan helps keep me cool while I walk.

Using it

My goal is 10,000 steps per day and I've found that I'm able to complete that in about three hours at a leisurely 2.1 MPH pace. The first few weeks I was pretty tired and didn't walk every day. It does take a period of acclimation to go from mostly sedentary to walking the equivalent of 2-3 miles a day so I wasn't kicking myself for not hitting my goal. Ultimately I want to build a long lasting healthy habit so I'm taking it slow and working up to my goal.

Some of my impressions after several weeks of daily usage:

I find that walking doesn't impede with my flow state. I'm able to write code or documents just fine while walking. I'm writing this blog post while walking. The constant pace and repetitiveness of it isn't distracting so it's quite easy to forget I'm on a treadmill after a while.
I usually spend about 2 hours on the treadmill in the mornings. More if I have meetings and need to stop walking so I can interact in the meeting. Ideally I'd block off time for walking in the mornings and is something I will be exploring. I've had a few times where my mornings were jam packed with meetings which caused me to miss my walk. Once I get going its hard to switch gears and move to the treadmill in the afternoons.
Walking can be distracting in meetings. For meetings I need to be highly engaged I usually stop the treadmill but stay standing. For low engagement meetings I'll keep walking. Walking and talking leaves me breathless so I usually need to stop if I enter a discussion. This should be less of a problem as I acclimate and get healthier.
I'm constantly drinking water while walking, no spills yet! Haven't tried drinking hot coffee yet, though I might just skip that.
I keep a pair of shorts, socks and my walking shoes next to the treadmill so I have no excuses to not walk.

Conclusion

I'm really enjoying my walking desk. I'm getting in physical activity where I'd otherwise be sitting at my desk. My work is getting done and I'm able to optimize my time outside work for other things I enjoy doing (like hanging out with my family)!

My key takeaways from this project:

If I expect myself to use this walking desk there needs to be almost no barriers to me hopping at any time.
Creating a comfortable setup (keyboard, trackpad, monitor, etc) that is very similar to my desk setup is important. If I tried to work off my laptop instead of with external monitors and peripherals like I normally do I think I would be way less motivated to hop over to the standing desk.
Walking is great!

If you've been considering a walking desk, I'd highly recommend trying it out. Just make sure you have the space to dedicate to it. You can always pick up a cheap used treadmill and rig up a desk setup.

Lessons learned from dabbling in the stock market

2021-12-27T00:00:00Z

I've dabbled in investing in the past but over the last two years I've really gotten involved heavily with managing my portfolio. I'm still a complete amateur but I did pick up some very valuable lessons:

I've started to attach probabilities to events way more than I used to. This has the benefit of helping me think about potential events more objectively.
There is a zen to riding the highs and lows of the market. It takes confidence in a long term game plan, diversification, and some good ole detachment in order to psychologically bear the swings.
I learned not to go into a trade without an exit plan (still working on actualizing this one).
The more you risk, the more you could potentially be rewarded.
You could also lose everything in one bad trade. Don't put all your eggs in one basket.
Leverage is a powerful and dangerous tool. Use it with extreme caution.
A stock's price usually reached an equilibrium until some catalyst occurs. The same could be said about life, usually we reach some steady state until a catalyzing life event occurs.
- TODO: explore how to introduce catalysts in a more consistent manner.
Companies rise and fall on a regular basis. What seems like a juggernaut one year may implode the next. Same for companies that seem to be worthless can be turned around and brought back from the grave.

I should have learned more bash sooner

2021-12-26T00:00:00Z

Bash tools are super powerful once you get a hang of using it. This whole blog is pretty much a bunch of scripts and Pandoc. Its pretty cool what you can build once you get over the learning curve.

I'm a few years into my career now and I've only really used shell scripts when working with CI. I've never felt totally comfortable with it, maybe it's because it feels like casting spells by uttering some obscure incantations? I mean look at one of the lines I put together to sort an array for this blog:

IFS=$'\n' SORTED_POSTS=( $(for j in "${POSTS[@]}"; do echo $j; done | sort -t ";" -k 1 -nr) )

What does all this mean?
IFS=$'\n' Hackily set the internal separator to newline so it stops breaking my strings up by spaces
$(for j in "${POSTS[@]}"; do echo $j; done | sort -t ";" -k 1 -nr) Loop through the $POSTS array (not shown) and sort by the first column delineated by ;
SORTED_POSTS=( ... ) Create an array from the output of the previous command

Hell, this probably isn't even the right way to do what I wanted it to do, but it works for my use case. What's really cool though is just how powerful and terse it is. The difficulty lies in getting started with it.

Some of the Pros and Cons I've encountered so far:

Pros

Portable between many different systems
Plenty of information available online
Very powerful and terse

Cons

Learning curve is steep
Lots of different ways to do X but somebody on the internet will tell you you're wrong
Readability is not very good. If you want to share scripts with a team, you might want to choose something like Python.

So if I were to advise my slightly younger self:

Take a few days and learn how to write bash scripts.
Read up on the basics of bash scripting.
- Learn X in Y seconds
- Bash Guide
Try writing some scripts to automate things (like setting up your dotfiles repo sooner)
If you find yourself reaching to some programming language to do something, try doing it in bash first.
Don't bother with anything complicated interactively. Create a script file and execute it if its anything more than a one liner.

2018 Reading List and Look Back

2019-01-01T00:00:00Z

Changes in 2018

Last year I left Penguin Random House in April to work at Noom. I’m normally a utility reader and don’t do much reading for entertainment. During my entire 2.5 years at Penguin Random House, I think read at most a handful of books.

The culture at Noom really embraces reading and encourages it, so this year I got some recommendations from my colleagues and began reading. This list is all business-y books that I have annotated with some thoughts. For the most part, they were all useful in their own way and once internalized, have helped me be more thoughtful and analytical.

Patrick’s 2018 Reading List

The Goal - Eliyahu M. Goldratt
The Phoenix Project - Gene Kim, Kevin Behr, George Spafford

The Phoenix Project is based on The Goal and references it often. Both weren’t particularly well written but they do give an interesting look at how manufacturing principals can be applied to modern office style work. An interesting way to think about how to structure teams and organize work.
The High Growth Handbook - Elad Gil

Quite a comprehensive look into what it takes to build and grow startups. A lot of good information for startup newbies like me. I wasn’t too into the interviews but I can see myself going back and re-reading this every so often.
Powerful - Patty McCord *Recommended

A look into how Netflix does things. I am still reading it but it’s quite a good read. Structured more as a guide for how to run a company the Netflix way than a tell all.
The Power of Habit - Charles Duhigg *Recommended

Interesting book on the psychology of habits, how to change them and how to re-wire our reward mechanisms. Not sure if I’ll try to apply any of the concepts to myself but it’s good information to internalize.
Clean Code - Robert C. Martin *Recommended

The only software development book I read this year. Classic advice for writing maintainable code.
ReWork - Jason Fried and David Hansson

A compilation of what seem like blog posts. A lot of advice on various things. I wasn’t too impressed by it, but it’s a quick read.

Health and Fitness

2018-12-29T00:00:00Z

I've been on quite the fitness journey these last few months. Here's my story.

How it started

I didn’t grow up athletic. I didn’t play any sports, exercise, or spend much time outdoors as a kid. I enjoyed indoor activities like video games, legos and computers. When I got into my 20s, I got into cycling and rode with friends for fun and as a way to commute to work. It was hobby first with beneficial exercise as a side effects.

Eventually I stopped cycling. I began eating more and more junk food and started gaining weight. My attention wasn’t on my health and over time the negatives began to manifest. I got fat, which caused back pain when I would sit for hours on end at work. My poor posture also exasperated any issues, and compounded any back pain. At night I would be listless, staying up late and waking up early for work. Overall the effect was just generally feeling crappy, lethargic, and moody. It was subtle, not huge changes to how I felt, which was the most insidious part. Getting unhealthy was a long drawn out process that became a normal part of my life.

Inspiration

I realized the benefits of being fit far outweighed the startup and maintenance cost of getting the habit in place. I was partly inspired by Wes Bos and Scott Tolinski in an episode of their developer focused podcast, Syntax where they discuss the benefits of working out and living a healthier lifestyle. At the same time, I also began to realize that there was a relationship between obesity and the impact it could potentially have on career opportunities in the future.

I realized that being overweight can be a negative signal in the professional world. There is a negatively skewed bias against overweight individuals in society where people associate poor self control, discipline, and other negative traits with obesity while subconsciously associating positive traits to fitter individuals. This led me to a theory which I call the “No Fat CEOs” theory (which is completely unsubstantiated and not backed up by any facts) where corporations are more frequently led by individuals who are not overweight. Being overweight and unhealthy in my opinion is a liability to my career and I wanted to turn my fitness from a liability to an asset (or at the very least have no effect).

As I slid into my 30s, I began noticing aches and pains and a general lethargy as I gained weight. Frankly, I felt like garbage. Walking up stairs would get me winded, bending over wasn’t easy. My body felt and looked like crap. The impact of being overweight on both my health and my psyche compounded the more overweight I became. It was easy to get stuck in a loop where I ate crap, felt like crap and looked like crap. I needed to break out the cycle.

Getting help

I had never set foot in a gym before and for a newbie like me, it can be quite a scary experience. I had no idea what exercises to do, how to do them and even where to start. I had learned recently that if you’re not an expert in something, the fastest and easiest way to obtain knowledge is to get help. So I hired a personal trainer and started going to a training session once a week.

My trainer talked to me about my fitness goals, created a program for me, taught me the exercises, and has kept me accountable. She helped me overcome my fear of going to the gym. Once I was comfortable, I set a goal of going to the gym at least 3 times a week. I made time during my week and treat going to the gym as a top priority in life.

Conclusion

Since undertaking this fitness journey, I’ve noticed numerous benefits:

I haven’t lost weight but I have lost fat and gained a lot of muscle.
My back doesn’t hurt any more.
I sleep better at night. Im going to bed earlier, sleep well, and wake up feeling rested.
I noticed I’m more productive at work, I feel more focused.
My posture is much better.
Overall I feel strong and healthy.
The feeling of accomplishment after a good session at the gym is itself a great reward and motivator.

There is a startup and maintenance cost to investing in my fitness and health, but I’ve already started collecting the dividends.

Deadlift 1 rep max at 300lbs with a goal of 400lbs by the end of 2019.

Things to keep in mind while writing a VS Code extension.

2017-11-07T00:00:00Z

This article was originally posted on medium.

VS Code is a great editor platform. I tried my hand at creating an extension
and although the documentation is fairly complete and thorough. I still
ran into some issues. I’m documenting them here for other extension
authors.

VS Code functionality takes the form of commands in the command palette.

Promises

If you’re used to the ES6 Promise style then VS Code’s promise style may
throw you off. Here is the newer style way to handle errors in promises
by chaining a catch method after a then method:

// ES6 PromisedoPromise().then(val => {  handleValue(val)}).catch(err => {  handleError(err)})

VS Code’s Thenable style promise then method takes two params.

Success handler
Error handler

// Thenable style promisesdoPromise().then(  val => { handleValue(val) },   err => { handleError(err) })

Make sure to handle these errors correctly or you’re going to have a bad time.

WARNING: Promise with no error callback:3 · Issue #11693 · Microsoft/vscodeVSCode Version: 1.4.0 OS Version: 10.10.5 I'm developing an extension where I'm using Promises (bluebird in this case)…github.com

Settings

It wasn’t immediately clear how to add settings to my extension. Turns
out, I was looking for the wrong word. Settings are actually call
configurations and are located within contributions. Here is a direct link to the documentation.

To add a setting to your extension, you need to add a key to your extension’s package.json file.

"contributes": {    "configuration": ...}

This configuration section takes a JSON schema that describes what your
setting should be. Basically what data type it takes, a default value
and a description.

It also wasn’t immediately clear how to implement a setting that takes an array of objects. You need to utilize the items key to describe nested values. Here is an example from my extension VSNotes:

Activation Events

If you have multiple commands you will need to make sure to include them in the activationEvents portion of your package.json. This tells vscode to execute the activate function in extension.js.

For example:

"activationEvents": [    "onCommand:extension.newNote",    "onCommand:extension.listNotes",    "onCommand:extension.setupNotes",    "onCommand:extension.openNoteFolder"],"commands": [    {        "command": "extension.newNote",        "title": "VSNotes: New Note"    },    {        "command": "extension.listNotes",        "title": "VSNotes: List recent notes"    },    {        "command": "extension.setupNotes",        "title": "VSNotes: Run setup"    },    {        "command": "extension.openNoteFolder",        "title": "VSNotes: Open Note Folder"    }],

It may not be necessary to put all your commands into the activation event depending on how you architect your application. This could be a
potential issue if commands fail to show up.

Complex commands

I have no idea what most of these do or why they are separate from the regular API. Could be because they are helper functions? Anyways, I used one to open a folder vscode.openFolder and it took a bit of time to figure out how to actually open a folder. If some functionality you need isn’t in the regular namespace API check the complex commands first.

Conclusion

Overall it was really painless to write a VS Code extension. The Yeoman scaffolding they provide really speeds up development time and makes the whole process much easier. Publishing is well documented and easy with the provided CLI tool. Overall a pleasant experience.

Check out the extension I wrote: VSNotes. It’s a quick way to set a note folder and create markdown notes right
from VS Code. It’s quick and painless and lets you leverage VS Code’s
extensions for your note taking workflow. Grab it from the marketplace.

A Website’s Life

2017-08-30T00:00:00Z

Imagine your job is to run a website.

You build it, get it running, promote it and people start using it. Things are going well! After this initial rush you settle down into maintenance and growth mode. There is no more mad scramble to add new features or scale the site. Users like your content and slowly tell their friends. Things are growing, albeit slowly.

First you add some site analytics because you want to see who’s doing what on your site. Great, now you have a bit of insight into user behavior. You hear that buying ads on Facebook is effective to drive traffic, so you set that up and install their tracking code on your site. Easy peasy! You spend a couple bucks here and there, a/b test some things and do a couple of small test ad-buys. It turns out that its pretty difficult to get users to click on your Facebook ads, so you try Twitter, Pinterest, Instagram, and Google Adwords, to test things out. Some things work but you’re seeing sporadic results. Meanwhile, your marketing manager is complaining that the site has too many tags and that you need to add a tag manager. Your site’s header looks like somebody fired a shotgun of bad code at it, so you listen and install a tag manager. Awesome, that made things a bit neater and now you can easily install tags with the click of a mouse!

The internet has been buzzing about exit intent and how its the new hotness. You need to capitalize it to retain users! It’s so easy to sign up for another AdTech company and install their JavaScript snippet in your tag manager, so there’s not even a second thought. Now when a user tries to do anything on your site, it will show them a popup modal and ask them to sign up for their email list. If they just want access to your content, they’ll have to click on a snarky cancel message. Haha! “No, you don’t want awesome content?”, they’ll love that! Oh yeah, while you were installing exit intent popups, it turns out that email lists are the best way of contacting your users according to an article you read in Fast Company. So you sign up for Mail Chimp and begin cranking out emails. They say it doesn’t matter what its about, just that its on a consistent schedule. Let’s get that content flowing!

People love your content and sometimes post about it on social media. Awesome! New marketing channels! So you create a Facebook page, a Twitter account, a Pinterest board, an Instagram account, a Snapchat account and whatever else the kids are dooting on these days. Well it turns out managing five social media accounts is a lot of work! Good thing there’s a SAAS out there to solve your problem, so you sign up for one and start dooting to your users. Oh, in the mean time, wouldn’t it be cool for people to be able to doot to links to your content to their friends? So you add some social media links to your site so they can easily share content! Brilliant!

Your boss comes by and wants to create more of a community feeling to your site, so you add a comments section with the leading comment section SAAS provider out there. It’s better than installing your own comment section because now users can sign up with Facebook or Twitter! Except they inject ads onto your site unless you pay them, so you pony up the monthly fee and pay them to host comments. Users begin commenting! Sweet!

Uh oh, people begin commenting on random pages that the site is really slow and doesn’t load well on mobile! Well crap, what can we do about that? Well, there’s this thing called Google Accelerated Mobile Pages that you heard about on CNET, let’s try that out. So your developer makes your site AMP enabled and you begin serving content via Google AMP. Now when users search for content on Google, they get a cached version of your site. Uh oh, but it also means they’re sharing mangled URLs on social media now. Well, we’ll deal with this later, we have more pressing matters right now.

Turns out you left some ads running on an obscure ad network and you racked up a couple thousand dollars in charges. Well that sucks, things must have gotten lost in the shuffle when you were testing ad platforms. You turn it off and go through each ad network to check for running ads. After ensuring no new ad campaigns are running, time to solve some other problems. Your last email campaign had an abysmal open rate, so you’ll need to troubleshoot that, but first you need to schedule some doots on social media. You start troubleshooting your email campaigns and it turns out your emails are going straight to the spam folder. Probably because people don’t like getting multiple emails per week from your site, explains the high unsubscribe rate too. You don’t understand, people are really engaged with your content! They’re commenting and sharing links… right?

Well…

The handy social sharing links you installed have only been clicked a handful of times. The fancy comments section is full of unrelated and low effort comments. Nobody seems to be talking to each other, just using it as a place to air their grievances about the slow load times and the incessant nagging exit intent modal. They still love your content, however it’s getting more and more difficult to reach it.

The site experience is starting to get bad, the console starts spitting out JavaScript errors because some of the SAAS services you installed went out of business. Browser updates begin breaking your site functionality (or so your developer tells you, it’s actually terribly written code). Things are beginning to grind to a halt so you do what any sane person does, decide to throw it all away and start fresh with a newly redesigned website.

You hire a top tier web agency to build the site (priced accordingly). They come highly recommended by the team in your sister company that just went through this process. They agile their way into a beautifully designed work of art, complete with full bleed high resolution images, quirky and cool widgets hidden in off-grid locations, carousels for days and a revamped homepage with megabytes on megabytes of images. It’s beautiful! The work is complete and they hand the site over to you and your team.

Imagine your job is to run this website…

Lessons learned from starting a business while working full time.

2017-08-19T00:00:00Z

I started working on Collate in December of 2016. I was dissatisfied with the note taking apps on the market and wanted a tool that would work for my needs. I had been toying with the idea and finally started writing code over the Christmas holiday. I had a basic prototype built with Electron, bootstrap and jQuery by February, then rebuilt it with Electron, Vue.js and Bulma from scratch because the code base was terrible. Today, Collate is profitable, has a couple hundred users, was featured on Lifehacker and is doing well by my standards. Here are some hard learned lessons from the last 8 months of working on Collate while maintaining a full time job:

Have a tight scope — I made the mistake of saying yes to everything and pretty soon Collate began heading in a direction I wasn’t happy with. Users will ask for features, it’s only natural. It’s extremely easy to bloat the scope of your project if you say yes to everything. Having an extremely tight scope of what your project does and provides is essential to focus the small amount of time you have to devote to the project. I’m working on the next iteration of Collate and it will be extremely pared down.
Use what you know — Use technology you know like the back of your hand. This advice is given a lot and for good reason. If you need to spend time learning a new piece of tech, it’s time away from making your product more stable. I use Wordpress in my day job, so it’s only natural I use it for my product’s site. It’s not sexy but it works.
If you have to learn new tech, experiment and choose what works for you — Other times you might not know how to use the right technology for the job. In these cases experiment and see what works best for your needs. I went with Vue.js for the second iteration of Collate because I was working on a project at work that utilized it. Using it in both places helped both projects immensely.
Charge money — This is one thing I’m glad I got right from the start thanks to advice from Indie Hackers and other startup founders. Charge money from the get-go and get customers comfortable paying for your product. Collate was profitable the second week out of release. The money gives you a cushion to experiment and grow the business. Since I’m working full time, all profits go back into the project to pay for things like servers, lawyers, and third party services I may use.
It’s okay to not respond to emails — Customer service is a very time consuming task. I made the mistake of replying to each and every email in the beginning, usually with long explanations of what I was doing and my plans. With limited time, choose your service requests carefully and try to minimize time spent on it. It doesn’t mean you don’t care, you’re just prioritizing. I got to a point where I would dread opening up my inbox, after I stopped responding to everything, things got a lot more manageable.
Get your legal ducks in a row — If you’re going to collect money for your project, spend a little up front as soon as you can afford to and get all your legal stuff squared away. I started my corporate entity (Collate, LLC) through a corporate filing company in my state. Opened a bank account with the same bank I use for my personal accounts. Then, got my End User License Agreement and Privacy Policy looked at by an attorney I found on Upcounsel.
Think about marketing — You don’t have to be a marketing wiz but consider it while building out the product. At the very minimum, start early by building a email list of people interested in your product. You don’t have to constantly market to your list but its useful to have people to notify when you have something to announce.
Pace yourself — I wish I had learned this lesson sooner. I worked on Collate furiously for the first six months, pumping out feature after feature and release after release. I was putting in close to 40 hours a week on nights and weekends working on Collate after work for a while. I got so burnt out that I had to step away from the project and try working on something else (and even not working on anything!). I’m back on it now, but working on it at a much slower pace. Take the time to decompress each day, get exercise and make sure to sleep.

Side projects while working full time are a lot of work but also very rewarding. The best part for me was building something from scratch that I’m passionate about building, something that I don’t get at my day job. It’s also pretty great to build something and then watch other people get excited about it.

From Community College to Columbia

2016-12-12T00:00:00Z

I dropped out of college at twenty. The next few years were spent at various jobs until I ended up as a legal assistant at a law firm. The firm was filled with fax machines, typewriters, and massive paper files. What was the norm in 1988 was still the norm in the legal industry at the time. It was a slow moving beast, mired in paper and yesterday’s technology. I wanted to work with modern technology and be part of the future but there was no future for an aimless college dropout. I couldn’t see a way forward, so I went back.

The Borough of Manhattan Community College

Community college was an eye-opening experience. On one hand you had the highly motivated students, usually older, who had experienced life first hand. Many had full time jobs, kids, and experience working low wage jobs. Most of the ones I met worked in retail or dead end jobs, and college for them was a ticket out of their daily struggle and into a comfortable future. On the other hand, you had the unmotivated students, usually younger and straight out of high school. Most treated it as an extension of high school, 13th and 14th grade, as a way to defer working for a little bit. While I could sympathize with them, I fell into the former group of students. I was motivated, but I lacked a clear picture of where I wanted to be. My idea of dreaming big was a degree from the local four year college and a modest job doing something better than what I had been doing at the law firm. At this point, I was doing well in school, working my way through a business management degree, content with my potential future. Who knew that taking a mandatory speech class, and randomly choosing a time slot that worked well with my schedule would land me with a professor that would change my life?

My Speech 101 professor, Dr. Poster, is a long time teacher and administrator at the Borough of Manhattan Community College (BMCC). She has a reputation of being a tough teacher, who expects a lot from her students and will hold them to a high standard. While plenty of her colleagues passed students who handed in sub par work, she held firm and expected quality work. She has an uncanny ability to see the best in her students, and to push them hard to realize their potential. Which is what she did with me. She brought me into her office one day and asked me about my plans for the future, what I wanted to do with my life, and where I saw myself going after I finished at BMCC. I gave her the rough layout of my college and career goals. Then she dropped a bomb on me.

“Why don’t you apply to Columbia?”

I had never even given Columbia serious consideration. That seemed way out of my reach, me, a college dropout in a no-name community college. I wasn’t a savant, I hadn’t done anything truly worthwhile, what chance did I have? She convinced me that it was worth trying and helped connect me to some people who could help with the application process. To my amazement, I got in.

Columbia University

Walking through the gates on 116th street and Broadway the first time was surreal. Stepping onto the Columbia grounds gives you the feeling that you’ve been transported miles away from New York City. I walked through rows of trees lining the main entrance which drew me into the center of campus. To the left were the stairs leading to Low Library, and to the right grassy fields and cleanly trimmed hedges leading to Butler Library. I can only describe the feeling as a sort of reverence, standing in this place where many generations of students have come and gone. A sort of… temple to the religion of academia. I had lived in New York City my entire life and had never known there was such a place.

My first semester there, I took a “relatively light” load of four classes while still working part time. Very quickly, I realized that four classes was not a light load and I had to quit my job. If I was going to pass my classes, showing up and doing homework was not enough. I was under water my first week of classes, drowning under the crushing weight of the volume, frequency, and difficulty of the work. It turns out that community college had not adequately prepared me for the rigors of real academia. Classes were much harder, especially since I was majoring in Economics-Mathematics, a major which combined the two disciplines and was similar to completing 80% of both degrees. To survive in this tough academic environment, you enter a sort of perpetual fight or flight mode. Fear had a lot to do with the stress. Fear of failing a class and extending my stay another semester with the associated costs. Fear of failing out of the school, leaving with significant debt without a degree to show for it. Fear that now that I had reached for the stars, I would fail and come crashing back to the ground. All these fears would be my main motivator and subsequently the reason I was able to graduate on time.

I’d best describe the time between when a semester begins and when it ends as a long slow grind. You’re grinding out work, studying, reading, writing and problem sets. Once you complete a task, work begins on the next one, over and over until it’s time to sleep. In community college the bar is set a lot lower, mostly because the bar of students is a lot lower. Teachers expected average work and students presented a wide range of quality. At Columbia, I noticed that the range of quality was a lot smaller, and the quality of work was a lot higher. This meant that in order to be an average student at Columbia, I needed to work harder than I’d ever worked at BMCC. I had friends who were top students at BMCC, who were accepted to Columbia at the same time as I was. They echoed similar sentiments about the intensity of the work. I had thought of myself as an average student at best, and here were people I had considered harder working and smarter than myself struggling to get by.

There was an aura of stress surrounding the student body. As my time at Columbia progressed, I noticed I started showing it as well. I don’t know if this was an experience unique to Columbia, top tier schools, or just college in general but I do know that during my time at BMCC I had never experienced that dreadful fear of failing a class. At BMCC the worst that could happen was that I would get a C. At Columbia the worst was failing a class and throwing away thousands of dollars as a result. On the flip side, however, I’ve never seen as many hard-working people in one place in my entire life. The maturity and discipline it took for these students to buckle down and study was inspirational for me. It helped to be surrounded by these hard-working students. I fed off the energy of my fellow students. I found it refreshing to be surrounded by people who were motivated especially coming from an environment where a fraction of your grade was determined by your attendance.

I noticed the disparity between the students the most during group assignments. Once at BMCC, I worked with a group to produce a final report which would make up a large portion of my grade. One of our group members disappeared part way through the semester and did not respond to any of our emails. Left in charge of her portion we had to split her work. On the final day of class, when we were slated to hand in our report, she showed up with a crumpled print-out with a paragraph which was supposed to represent her portion, fully expecting us incorporate it with our work. Baffled, we had to turn her down and let her know that her work was unacceptable. In comparison, at Columbia, I worked with a group to present a report in which we all agreed to go the extra mile and create a video showcasing our findings. All members of the group were active, engaged, and contributed beyond their fair share of work. Though this is only anecdotal evidence, I found comparisons between idiosyncratic differences in the student bodies different enough to be startling.

However, the student body was not without its flaws. Though racially diverse, the students at Columbia mostly came from the same privileged socioeconomic background. I found the school’s population to be homogenous. I was used to interacting with students with working class backgrounds from New York City. The difference was quite stark, and I found myself having a difficult time forming friendships with my peers. I naturally gravitated towards older, “unconventional” students, who were also returning to college, mostly military veterans. I didn’t have much time to socialize though. I had a carefully planned course schedule to adhere to if I was going to graduate on time. I stuck my nose in my books and worked harder than I had ever worked in my life.

Post Columbia

I graduated. It surprised me much more than it should have. In hindsight it wasn’t all just studying and stress, there were plenty of good times along with the bad. I met a lot of great people during my time, many who have gone on to do some impressive things. I had worked my way through two and a half years and landed an internship which eventually turned into a job. From there, I transitioned to my current role as a Data Analyst at Penguin Random House, the largest book publisher in the US. Without my education from Columbia, data analytics would be a complete mystery to me. Putting in the hours and studying outside of work isn’t a chore as it would have been prior to going back to school. An example of that is that since graduating, I’ve been able to teach myself programming. I had attempted to do so in the past but had never pushed through the tedious parts, always getting stuck. Also, thanks to the fundamental math and economic education I received, I find working with large data sets much easier.

Looking back at it now, I can appreciate the rigor and the intensity of the work. Columbia helped push my boundaries and helped me realize what I’m capable of doing. Just three years prior, I matriculated from BMCC, content with a future that was just slightly better than the one I left. However, I would never have even thought to reach so high had Dr. Poster not seen something in me and pushed me to apply. It’s truly amazing what an effect one person can have on the trajectory of another person’s life and I’m grateful to her for what she’s done for me. She made me promise that I would never remove BMCC from my resume, to show others what students from a community college could achieve. I leave it on there for her, and to remind myself of my roots.