Every time an entity gets referenced, whether a person, a project, a concept, Mem0 increments a mentions count on that node and its edges. It’s tracked on every update. Written to the database. Never read back. Never used in retrieval scoring. The data’s there, the infrastructure’s there — it just hasn’t been connected yet.

That turned out to be the recurring theme. These codebases are full of good infrastructure and good ideas, often in different places than you’d expect.

I should back up.

Why we did this

We’re building Mneme, a proactive engineering intelligence system, where it watches your team’s signals (commits, PRs, Slack threads, Jira tickets, error spikes, stripe txns) and synthesizes context before decisions get made. To build that, we needed to understand what already exists in the memory space. Blog posts or READMEs only get you so far, so we dove into the actual source code.

So we cloned 10 of the most relevant repos and read through them (read: with some heavy lifting from Claude Code). The list~

1. Graphiti (23K stars),

2. Mem0 (48K stars, $24M raised),

3. Cognee (12.5K stars, $7.5M seed),

4. Letta (21K stars),

5. memU (11K stars),

6. SimpleMem (3K stars),

7. mcp-memory-service (1.4K stars),

8. A-Mem (800 stars),

9. memsearch (612 stars), and

10. claude-cognitive (310 stars)

Over 120K GitHub stars and $31.5M in venture funding aimed at a single thesis: LLMs need persistent memory to be useful.

They’re right about the thesis. The disagreement is about how to get there. And we have obvious bias since we’re building in this space. The research is real, but the interpretation is ours. Keep that in mind.

Three approaches

The approaches fall into three paradigms~

1. System-managed extraction: the system decides what to remember

   1. Mem0, Graphiti, Cognee fall into this bucket: expensive writes, clean output

2. Agent self-management: the agent decides what to remember

   1. Letta’s MemGPT holds its own here - elegant, and depends on agent discipline

3. Compression and retrieval: the system compresses conversation history

   1. SimpleMem, memsearch, mcp-memory-service, claude-cognitive - all trade fidelity for token efficiency

Most production systems will need some mix of all three. But what I found more interesting than the paradigm choices was what happened when I looked at the actual implementations.

What we found when we read the code

The gap between marketing and implementation runs in both directions. Both directions are worth seeing.

Graphiti’s bi-temporal model is as good as advertised. This is from graphiti_core/edges.py:

class EntityEdge(Edge):
    # Edge base class provides: uuid, source_node_uuid, target_node_uuid, created_at
    name: str = Field(description='name of the edge, relation name')
    fact: str = Field(description='fact representing the edge and nodes that it connects')
    fact_embedding: list[float] | None = Field(default=None)
    episodes: list[str] = Field(default=[])
    expired_at: datetime | None = Field(
        default=None, description='datetime of when the node was invalidated'
    )
    valid_at: datetime | None = Field(
        default=None, description='datetime of when the fact became true'
    )
    invalid_at: datetime | None = Field(
        default=None, description='datetime of when the fact stopped being true'
    )

Four datetime fields. valid_at and invalid_at track when a fact was true in the real world. created_at and expired_at track when the system learned and retired the fact. When a new fact contradicts an old one, the old edge gets invalid_at set to the new edge’s valid_at. Nothing is ever deleted. This enables a query you can’t do anywhere else: “What did we know about X at time T?”

Read more

The problem is deceptively simple: when I git push for any of my various open source projects, I would like it routed through my personal account.

And when I push from work, it should use my work email. No manual switching. No “oh crap, wrong email” moments at 2am.

Here’s the setup I landed on. It took a couple minutes and I haven’t thought about it since.

Note: If you’re an SSH person, you can solve this with separate keys and ~/.ssh/config Host aliases but I use HTTPS across machines and prefer leaning on gh’s token management rather than juggling key pairs

The Commit Identity Problem

Git uses a global user.name and user.email from ~/.gitconfig for every commit. If your global config is your work email, every commit across every repo — including your weekend passion project — gets stamped with you@acme.com.

You could set a per-repo override with git config user.email in each project dir, but that’s fragile. You’ll forget (read: I have). You’ll clone a new repo and push three commits before realizing you’ve been committing as the wrong person.

The Fix: includeIf with Remote URL Matching

Git v2.36 introduced hasconfig:remote,1 which lets you conditionally include config files based on the remote URL. This is the key.

First, create a separate config file for your personal identity:

  # ~/.gitconfig-personal
  [user]
      name = Your Name
      email = you@gmail.com

Then add conditional includes to your ~/.gitconfig:

  [user]
      name = Your Name
      email = you@work.com
  [includeIf “hasconfig:remote.*.url:https://github.com/YourPersonalUsername/**”]
      path = ~/.gitconfig-personal
  [includeIf “hasconfig:remote.*.url:git@github.com:YourPersonalUsername/**”]
      path = ~/.gitconfig-personal

That’s it. Any repo with a remote pointing to your personal GitHub org automatically uses the personal identity. HTTPS and SSH both covered. No per-repo config, no directory-based hacks.

The Push / Pull Auth Problem

Solving the commit identity is only half the battle. When you git push, git also needs to authenticate with GitHub. If you use gh (the GitHub CLI), it has its own auth — and it only has one “active” account at a time.

The naive approach: gh auth switch before every push. The correct approach: a custom credential helper that routes automatically.

The Fix: A Custom Credential Helper

Git’s credential system supports custom helpers — scripts that receive the target host and repo path, and return the appropriate credentials. We can write one that checks the repo owner in the URL and picks the right `gh` token:

  #!/bin/bash

  # ~/.local/bin/gh-credential-switcher
  if [ “$1” != “get” ]; then
    exit 0
  fi

  host=”“
  path=”“

  while IFS=’=’ read -r key value; do
    [ -z “$key” ] && break
    case “$key” in host) host=”$value” ;; path) path=”$value” ;;
    esac
  done

  if [[ “$host” != “github.com” ]]; then
    exit 0
  fi

  if [[ “$path” == YourPersonalUsername/* ]]; then
    user=”YourPersonalUsername”
  else
    user=”your-work-username”
  fi

  token=$(gh auth token --user “$user” 2>/dev/null)
  if [ -n “$token” ]; then
    echo “protocol=https”
    echo “host=github.com”
    echo “username=$user”
    echo “password=$token”
  fi

Make it executable chmod +x) and wire it into your ~/.gitconfig:

  [credential “https://github.com”]
      helper =
      helper = !/path/to/gh-credential-switcher
      useHttpPath = true

The empty helper = line clears any previously registered credential helpers (looking at you, macOS Keychain). And useHttpPath = true is critical — without it, git only sends the host to credential helpers, not the repo path, and the script can’t distinguish between orgs.

The macOS Gotcha

If you’re on a Mac, Xcode ships a system-level gitconfig at /Applications/Xcode.app/Contents/Developer/usr/share/git-core/gitconfig that registers osxkeychain as a credential helper. This runs before your custom helper and returns whatever cached credential it has — defeating the entire setup.

The helper = line in [credential “https://github.com”] clears this. If it doesn’t stick (system-level configs can be stubborn), you can also add a global [credential] block:

  [credential]
      helper =
      helper = !/path/to/gh-credential-switcher

Prerequisites

Both GitHub accounts need to be logged into gh:

  gh auth login  # login with your work account
  gh auth login  # login with your personal account

You can verify both are available with

gh auth status

The “active” account doesn’t matter anymore — the credential helper bypasses it entirely.

Verification

Quick sanity check across repos:

# Personal repo
$ git -C ~/projects/side-project config user.email
you@gmail.com

$ git -C ~/projects/side-project ls-remote --heads origin
a1b2c3d  refs/heads/main

# Work repo

$ git -C ~/projects/work-thing config user.email
you@work.com

$ git -C ~/projects/work-thing ls-remote --heads origin
d4e5f6a  refs/heads/main

Different identities, different auth tokens, zero manual switching.

TL;DR

1. Layer

   1. Mechanism

   2. Config

1. Commit identity

   1. includeIf hasconfig:remote

   2. ~/.gitconfig + ~/.gitconfig-personal

2. Push / pull auth

   1. Custom credential helper

   2. ~/.local/bin/gh-credential-switcher

3. macOS Keychain bypass

   1. helper = (clear line)

   2. ~/.gitconfig

4. Path-based routing

   1. useHttpPath = true

   2. ~/.gitconfig

Clone any repo and it just works. New personal project? Personal identity and auth. New work repo? Work credentials. The setup is entirely URL-based, so it doesn’t matter where on disk the repo lives or what branch you’re on.

Ten minutes of config. Months of not thinking about it.

As an engineer, I often find myself sifting through GitHub pull requests, pulling branches down and opening .diffs to review changes. It’s a tedious process to understand the full context and then provide helpful comments. After repeating this for months, why not automate everything with a bookmarklet?

ChatGPT 4o is powerful enough to give me summaries of PRs but selecting all the text on the site isn’t good enough and introduces a lot of clutter. It is much better to provide a diff with appropriate context (like full changed source files) so it can paint a clearer picture.

This also helps when I’ve finalized my own PR and want to provide useful pointers on what’s changed. I feed the diff into the LLM along with the GitHub PR template and it spits out the changes instead of having to manually type it out.

Step 1: The Simple Idea – just open the .diff

GitHub already provides .diff and .patch files for PRs — by appending the extension to the URL, you get the all the changes in a raw format.

javascript:window.open(window.location.href + '.diff');

What if I'm not on a PR page? I also needed the bookmarklet to know when I was on a valid pull request page and act accordingly.

Step 2: Handling Edge Cases (Because I Click Everywhere)

I quickly realized that I’m not always on the "Files changed" tab or even on a valid PR page. Sometimes I’d hit the bookmarklet while still on the main repo, the list of PRs, or even issues tabs. I needed to handle these cases gracefully.

So, I added checks for specific pages like /pulls, /issues, and /actions, displaying an alert if I wasn’t on a valid PR page. That made things a lot smoother:

if (/\/(pulls|issues|actions)|github\.com\/[^\/]+\/[^\/]+$/.test(window.location.href)) {
  alert('You are not on a valid PR page. Please navigate to a specific PR.');
  return;
}

Update: This guard evolved over time while building out the chrome extension to guard against all pages that do not follow the format of —

https://github.com/<repo-owner>/<repo-name>/pull/<number>

Step 3: Adding the "View raw" for Files

Once I had the diff part working, the next natural step was to extend the bookmarklets capabilities by opening each changed file in the pull request. This would be helpful in painting a better picture on how the files related to each other and to create a contextual mind map. The dropdown was just a click away, but why click it manually when I can automate that too?

The script was updated to automatically click the ellipsis buttons for each file, wait for the dropdown to appear, and then click on the "View raw" menu section. Here’s what it ended up looking like:

document.querySelectorAll('.file-header').forEach((fileHeader, index) => {
  const kebabButton = fileHeader.querySelector('.octicon-kebab-horizontal');
  if (kebabButton) {
    setTimeout(() => {
      kebabButton.closest('summary').click();
      setTimeout(() => {
        const viewRawLink = fileHeader.closest('.file').querySelector('a[href*="/raw/"]');
        if (viewRawLink) window.open(viewRawLink.href, '_blank');
      }, 500);
    }, index * 1000);
  }
});

Learnt the preferred nomenclature is kebab not ellipsis. Now the bookmarklet could handle both the .diff file and individual "View raw" links for all files in the "Files changed" tab.

The final code

javascript:(function() {
  const prUrl = window.location.href;

  // List of invalid URL patterns (main page, /pulls, /issues, /actions)
  const invalidPatterns = ['/pulls', '/issues', '/actions', /github\.com\/[^\/]+\/[^\/]+$/];

  // Check if the URL matches any invalid patterns
  if (invalidPatterns.some(pattern => prUrl.match(pattern))) {
    alert('You are not on a valid PR page. Please navigate to a specific PR.');
    return;
  }

  // Part 1: Open the diff file after extracting base PR URL
  const normalizedUrl = prUrl.split('/pull/')[0] + prUrl.match(/\/pull\/\d+/)[0]; // Extract base PR URL
  const diffUrl = normalizedUrl + '.diff';
  window.open(diffUrl, '_blank');

  // Part 2: If not on the "Files changed" tab, exit after opening the diff
  if (!prUrl.includes('/files')) {
    return;
  }

  // Part 3: If on the "Files changed" tab, open the "View raw" links
  const fileHeaders = document.querySelectorAll('.file-header');

  function openRawView(fileHeader) {
    const kebabButton = fileHeader.querySelector('.octicon-kebab-horizontal');

    if (kebabButton) {
      // Click the ellipsis button to open the dropdown
      kebabButton.closest('summary').click();

      // Wait for the dropdown to appear, then click "View raw"
      setTimeout(() => {
        const viewRawLink = fileHeader.closest('.file').querySelector('a[href*="/raw/"]');
        if (viewRawLink) {
          window.open(viewRawLink.href, '_blank');
        } else {
          console.error('Could not find the "View raw" link.');
        }
      }, 500); // Delay to allow the dropdown to appear
    } else {
      console.error('Could not find the kebab button.');
    }
  }

  // Loop through each file header, trigger the process for "View raw"
  fileHeaders.forEach((fileHeader, index) => {
    // Stagger the openings to prevent overload
    setTimeout(() => openRawView(fileHeader), index * 1000);
  });
})();

And a One-Liner Bookmarklet

javascript:(function(){const prUrl=window.location.href;if(/\/(pulls|issues|actions)|github\.com\/[^\/]+\/[^\/]+$/.test(prUrl)){alert('You are not on a valid PR page. Please navigate to a specific PR.');return;}const diffUrl=prUrl.match(/\/pull\/\d+/)?prUrl.split('/pull/')[0]+prUrl.match(/\/pull\/\d+/)[0]+'.diff':null;if(diffUrl)window.open(diffUrl,'_blank');if(!prUrl.includes('/files'))return;document.querySelectorAll('.file-header').forEach((fileHeader,index)=>{const kebabButton=fileHeader.querySelector('.octicon-kebab-horizontal');if(kebabButton){setTimeout(()=>{kebabButton.closest('summary').click();setTimeout(()=>{const viewRawLink=fileHeader.closest('.file').querySelector('a[href*="/raw/"]');if(viewRawLink)window.open(viewRawLink.href,'_blank');},500);},index*1000);}});})();

You can add this by either

1. using this website or

2. bookmarking this site, changing the name to GH - ChatGPT or any equivalent, editing the bookmark and pasting the one-liner above in the URL section

Remember to always allow pop-ups when using this for the first time and try not use this on PRs with many files changed, it’ll open a bunch of tabs.

Another one - Opening a ChatGPT chat programmatically

If you don’t want to copy the contents of the diff into ChatGPT you can run this bookmarklet which opens a new chat. You do need an account —

Code

(function() {
  const prUrl = window.location.href;

  // Check if the URL contains ".diff" to ensure it's a valid diff page
  if (!prUrl.includes('.diff')) {
    alert('This bookmarklet only works on .diff pages.');
    return;
  }

  // Get the contents of the diff file
  const diffContent = document.body.innerText;

  // Create the prompt for ChatGPT, broken into multiple lines for readability
  const prompt = encodeURIComponent(
    `You are an expert programmer in React Native and Python. I've provided a diff file.\n\n` +
    `I want you to summarize the changes and provide any helpful comments, feedback, fixes, and optimizations that can be done to the code to make it better in bullet points.\n\n` +
    `I can also provide the files if needed, let me know.\n\n` +
    `Here is the diff:\n\n${diffContent}`
  );

  // Open ChatGPT with the pre-filled prompt
  window.open(`https://chatgpt.com/?q=${prompt}`, '_blank');
})();

And a One-Liner Bookmarklet

javascript:(function(){if(!window.location.href.includes('.diff')){alert('This bookmarklet only works on .diff pages.');return;}const prompt=encodeURIComponent(`You are an expert programmer in React Native and Python. I've provided a diff file. I want you to summarize the changes and provide any helpful comments, feedback, fixes, and optimizations that can be done to the code to make it better in bullet points. I can also provide the files if needed, let me know.\n\n${document.body.innerText}`);window.open(`https://chatgpt.com/?q=${prompt}`,'_blank');})();

Same instructions as above to create a bookmarklet. Happy Friday!

Chrome Extension

Adding bookmarklets is tough. I created a chrome extension that streamlines the workflow for interacting with GitHub pull requests. This tool allows one to quickly open diff files, access raw files, and even send pull request diffs to ChatGPT for feedback—all with the click of a button! The extension is built with TypeScript for type safety and leverages Chrome's powerful scripting and activeTab permissions to interact directly with the GitHub page.

The best part? The tool is fully customizable! I added an editable prompt feature for ChatGPT, so one can fine-tune the kinds of feedback they’re seeking. From improving code quality to reviewing changes more efficiently, this extension has been a game changer in my development workflow. If you prefer a GUI, there is an open source chrome extension available here —

• https://github.com/VirenMohindra/github-diff, and here

• https://chromewebstore.google.com/detail/github-diff/hhopppbhdogliepfdaihcoicedlieibn

Premise

Sports books in America are synonymous to the credit card companies of yesteryear. They’re looking to capture the largest market share in the least amount of time, and are willing to cut into their massive profits to make the onboarding process easier.

This aggressiveness lends them to providing outrageous sign up bonuses for first time users, and follow up bonuses so you don’t churn away. Let’s take a look at our players.

1. BetMGM - There are many benefits when you’re working with quite possibly the worst sportsbook user interface. Their

2. Fanduel - $3000 promo for Super Bowl. I’ve churned this twice for a neat 1.6K profit. My accounts been marked as a whale with my weekly turnover.

3. Hard Rock Casino - Can’t multi state bet which sucks since I really enjoyed turning a tidy profit on Alcaraz’s game.

4. Caesar’s - we’ll touch on this beauty later.

A conversation

Read more

Unanticipated Service Downgrades and Billing Confusion: Our journey into the abyss began with an unexpected downgrade in our Pro Bundle Plan. No communication, no email alerts – just a sudden change that we stumbled upon. This wasn't just an oversight; it reflected a deeper problem in transparency and customer communication within Magic.

The Domino Effect on Our Services: This downgrade triggered a loss of auto-refresh magic sessions. Imagine our shock when this led to a month-long (and counting) disruption of service for all our users. An app-breaking issue, stemming from a change we weren't even informed about!

Quick Fixes, Lingering Problems: Magic's response, while prompt, was akin to placing a band-aid on a gaping wound. The fixes resolved nothing substantial, leaving us to grapple with TypeScript issues and untested Expo packages. Our engineering team spent precious time scouring GitHub issues, trying to discern if the problem was on our end or Magic's. Spoiler alert: it was on Magic's.

Wasted Resources and Stalled Development: The fallout was immense. We burned through a long weekend and three full working days, troubleshooting, attending calls, and reporting issues – all to no avail. Our development cycle ground to a halt, as Magic's issues blocked us from pushing out new builds.

Reluctance to Change, Despite Discontent: Staying with Magic wasn’t a choice driven by satisfaction but a lack of bandwidth to switch providers or develop our own authentication solution. This is a crucial point for startups to consider – sometimes the devil you know seems better than the devil you don't.

Support Channels: A Labyrinthine Endeavor: Seeking support from Magic turned into a Herculean task. Immediate help was a myth; we were funneled into scheduled calls and endless back-and-forths, leading nowhere but to increased frustration.

A Broken Website Experience: To add insult to injury, Magic's website was a nightmare. A non-functional search feature, a user export request that took months to implement (and still underperforms), and the inability to perform basic sorting – it was less magic and more tragic.

Conclusion: A Call for Accountability and Change: This post isn't just a venting session but a call to action. Service providers like Magic must understand the gravity of their shortcomings and the ripple effect on their clients' businesses. As for fellow entrepreneurs and tech enthusiasts, let this be a reminder to continuously evaluate your service providers, demand transparency, and always have a contingency plan.

In the end, the magic we seek in technology services lies not in grand promises but in reliability, effective communication, and a commitment to problem-solving. It's time for Magic to step up their game or step aside.

The optimal use case is when you have access to OTA and can target prior versions to upgrade backend endpoints so that everyone is pushed to the latest version. With expo this fairly easy and can be run with this command

eas update --branch production -p ios

Code snippet below -

// your various imports here

// constants
const LATEST_VERSION = '2.2.8'
const LATEST_VERSION_NUMBER = Number(STABLE_VERSION.split('.').join(''))
const appStoreName = 'gently-shop-fashion-deals'
const appStoreId = 'id6450376418'
const appStoreURL = `itms-apps://apps.apple.com/id/app/${appStoreName}/${appStoreId}?l=id`

const Root = () => {
  const ver = Number(Constants.expoConfig.version.split('.').join(''))

  if (ver < LATEST_VERSION_NUMBER) {
    Alert.alert(
      'New Update Available',
      'For the best experience and new features, please update your app now.',
      [{ text: 'Update Now', onPress: () => Linking.openURL(appStoreURL) }]
    )
  } else
    return (
      <SessionProvider>
        <HomeStateProvider>
          <GlobalStateProvider>
            <StripeProvider
              publishableKey={STRIPE_PUBLISHABLE_KEY}
              urlScheme="gently"
              merchantIdentifier="merchant.com.gently"
            >
              <ThemeProvider theme={theme}>
                <Toast>
                <App />
              </ThemeProvider>
            </StripeProvider>
          </GlobalStateProvider>
        </HomeStateProvider>
      </SessionProvider>
    )
}

A simple explanation of the constants for the uninitiated

1. LATEST_VERSION: This will be the minimum semver in a string format. Any version below will be forced to update, i.e. a user still on v2.6.0 will see the non-dismissible Alert dialog if the latest app version is v2.8.0.

2. appStoreName, appStoreId: You can find these when visiting your app URL and copying it over. Various resources available.

Read more

One can ostensibly create a network graph of their most relevant potential users based on a few curl requests to Clubhouses undocumented API.
Disclaimer: I do not support reverse engineering any systems without authorized permission. This was purely an academic exercise.

Purpose

Finding users who are interested in your product is hard. Scratch that — finding users is hard. Clubhouse simplifies this by allowing their users to follow topics they are interested in, and joining clubs they fancy. Unfortunately clicking around on their mobile-only platform is a waste of time. I wanted a way to get the public information of Clubhouse users without manual work.

Armed with this information it should be trivial to create a master list of your target demographic who have specific interests, say journalists interested in improving their marketing skills.

I haven’t been timed out by Clubhouse yet.

Requirements

1. You’ll need a clubhouse account (which requires a verified phone number).

2. mitmproxy (optional) — All requests need to be authorized with a bearer token which can be grabbed via mitmproxy and tunneling through your PC as proxy. For the sake of conciseness I’ve chosen to not include them, finding the auth_check calls are trivial. From my understanding it is best to do this on your mobile since their might be additional tracking information and one might be painted as a bad actor.

Technicals

There are many additional headers and parameters which I’ve omitted for brevity. One can include those if needed, but I’ve included the bare minimum below to get a successful response from the servers.

POST | Get Your Profile

curl --location --request POST 'https://www.clubhouseapi.com/api/me' \
--header 'Authorization: Token {{YOUR_TOKEN_HERE}}'

The token should persist but refresh_tokens are also provided in the response.

All the relevant data should be under user_profile, in this format:

"user_profile": {
    "user_id": 578458,
    "name": "Viren Mohindra",
    "photo_url": "https://d14u0p1qkech25.cloudfront.net/578458_38c43e6d-b258-4301-9849-9f7748837880_thumbnail_250x250",
    "username": "virenmohindra"
},

GET | Get Club Members by Club ID

Required: club_id needs to be passed in as a parameter.

curl --location --request GET 'https://www.clubhouseapi.com/api/get_club_members?club_id={{CLUB_ID}}' \
--header 'Authorization: Token {{YOUR_TOKEN_HERE}}'

Response will be a list of Users, in this format:

{
    "user_id": 532095,
    "name": "Professor",
    "photo_url": "https://d14u0p1qkech25.cloudfront.net/532095_8a6db39e-7da0-4938-ab4c-1ab37c2f73cd_thumbnail_250x250",
    "username": "jocelynjkopac",
    "bio": "MY CLUBHOUSE BOOK JUST LAUNCH!!! 🎉🎉🎉 \nIt's on Amazon: Social Audio Jumpstart ✊🏾\n…",
    "is_admin": true,
    "is_leader": false,
    "is_member": true,
    "is_follower": false,
    "is_pending_accept": false,
    "is_pending_approval": false
},

POST | Get User by User ID

Required: user_id needs to be passed in as a parameter.
Note: Information needs to be x-www-form-urlencoded in the body.

Sometimes this particular request will hang if you haven’t provided the logging_context. If you’re following along with mitmproxy you might just have a JSON object handy. Use this nifty StackOverflow answer to convert the JSON object into a properly formatted value.

curl --location --request POST 'https://www.clubhouseapi.com/api/get_profile' \
--header 'Authorization: Token {{YOUR_TOKEN_HERE}}' \
--data-urlencode 'user_id={{USER_ID}}'

The most important fields would be the user_profile which houses all verified social handles and the list of clubs and topics.

An example sanitized response can be found as a Github Gist here.

Good luck!

Feature

You’re in a subpage on your website and you need to set SEO-related tags based on the information you receive from the back-end. For instance, if a user clicks on an events page, the title tag should expectedly display the eventName, and hopefully relevant information like the artistName and location, formatted for Facebook and Twitter (or any other relevant Open Graph protocol).

Problem Statement

You’ve tried hardcoding the meta tags, like such:

<title>Donda - Kanye West in Pittsburgh '21</title>

but instantly some issues pop up, namely: 1

1. The data isn’t getting propagated, and/ or is missing.

2. Duplicated tags (since you have base meta tags for the root page).

Fix

1. Customized <Head/> component removed from _document.js, and instead imported into _app.js

2. Each tag has a key prop which handles deduplication.

3. calling getServerSideProps inside the relevant page.

Explanation:

1. _document.js doesn’t support the title tag and I’d preferably like to keep all my tags in one single source of truth. 2 The easiest way to do this would be to create a wrapper component which renders all the base meta tags.

2. Each tag also a key property which will handle potential duplication. I used an enum to handle the various meta tags, so I wouldn’t mistakenly misspell any attributes. I’ve added them below for posterity:

enum MetaTagKeys {
  TITLE = 'title',
  META_TITLE = 'meta_title',
  DESCRIPTION = 'description',
  OG_TYPE = 'og:type',
  OG_TITLE = 'og:title',
  OG_URL = 'og:url',
  OG_DESC = 'og:description',
  OG_IMG = 'og:image',
  TWITTER_CARD = 'twitter:card',
  TWITTER_URL = 'twitter:url',
  TWITTER_TITLE = 'twitter:title',
  TWITTER_DESC = 'twitter:description',
  TWITTER_IMG = 'twitter:image',
}

An example of a meta tag in <DocumentHead /> would look something like this:

<meta name={MetaTagKeys.DESCRIPTION} key={MetaTagKeys.DESCRIPTION} content="Kanye West performs Donda live at the PPG Paints Arena in Pittsburgh on 10/27/2021."/>

This wrapper also contains the various scripts (dangerouslySetInnerHTML), and link icons which inevitably end up in your head tag. I prefer to keep _app.js as clean as possible.

3. To get Facebook, Twitter, and any other social network to scrape your page and prettily display your meta tags and content, one has to pre-render the page. In Next.js this can be done via getServerSideProps. 3

The final code in all its glory.

import Head from 'next/head'
import { GetServerSideProps, GetServerSidePropsContext } from 'next';

interface MetaTag {
  property: string,
  key?: string,
  content: string,
}

export const getServerSideProps: GetServerSideProps = async ({req}: GetServerSidePropsContext) => {
  const {url = ''} = req;
  const urlSlug = url.split('event/')[1];
  const eInfo = await User.getUserInfo(urlSlug);

  const metaTagTitle = `${eInfo.eventName} -  ${eInfo.artistName} in ${eInfo.eventState}`;

  const metaTagDescription = `${eInfo.artistName} performs live at the ${eInfo.eventLocation} in ${eInfo.eventState} on ${eInfo.eventDate}`;

  const metaTagsList: MetaTag[] = [
    // <meta property="title ... /> has the same property as the <title name="title>...</title>, but still needs a unique key.
    // we handle this logic while mapping over the metaTagsList prop.
    {property: MetaTagKeys.TITLE, key: MetaTagKeys.META_TITLE, content: metaTagTitle},
    {property: MetaTagKeys.DESCRIPTION, content: metaTagDescription},
    // <!-- Open Graph / Facebook -->
    {property: MetaTagKeys.OG_URL, content: url},
    {property: MetaTagKeys.OG_TITLE, content: metaTagTitle},
    {property: MetaTagKeys.OG_DESC, content: metaTagDescription},
    {property: MetaTagKeys.OG_IMG, content: "https://jpt-ugc.s3.ap-southeast-1.amazonaws.com/metaTag.png"},
    // <!-- Twitter -->
    {property: MetaTagKeys.TWITTER_CARD, content: 'summary_large_image'},
    {property: MetaTagKeys.TWITTER_URL, content: url},
    {property: MetaTagKeys.TWITTER_TITLE, content: metaTagTitle},
    {property: MetaTagKeys.TWITTER_DESC, content: metaTagDescription},
    {property: MetaTagKeys.TWITTER_IMG, content: "https://jpt-ugc.s3.ap-southeast-1.amazonaws.com/metaTag.png"}
  ];
  return {
    props: {
      metaTagsList
    }
  }
}

const Home = ({metaTagsList}: HomeProps): JSX.Element => {
// one might have a useEffect() hook here to get additional information or to set the title.

  const metaTagTitle = eInfo && eInfo.displayName !== null ?
    `${eInfo.eventName} -  ${eInfo.artistName} in ${eInfo.eventState}` : null;

return (
          <Head>
            <title
              key={MetaTagKeys.TITLE}>
                {metaTagTitle}
              </title>
            {metaTagsList
             && metaTagsList.map((entry: MetaTag) => (
               <meta
                 property={entry.property}
                 key={entry.key ? entry.key : entry.property}
                 content={entry.content}
               />
            ))}
       )
}

Another StackOverflow solution for reference.

But I’ve always found it difficult to navigate to the first commit. I do this for several reasons, whether I need to judge the age of the software — which I can also trivially do by checking the commit date of the README.MD or LICENSE, but is sometimes inaccurate. Or I’m curious, and I want to see how the project grew from the first sown seed. Barring quickly looking at the number of stars and forks of a project, I would hope that Github adds a label that can indicate when the project’s first lines of code were written.

Now, power users can click on the total number of commits on the homepage of a repository and go to the last page by simply editing the number after the SHA in the URL. An example:

TrumpTracker has 330 commits at the time of writing. The URL for the repo is —

https://github.com/TrumpTracker/trumptracker.github.io/commits/master?after=7d27f3b4278ad9a8f1282e0cc7dad53e9bf11c77+300&branch=master

Where the bolded 300 should be the

total # of commits - 30 === 330 - 30 = 300

And the constant 30 is the arbitrary number of commits they show per page.

One can perform these calculations and brute force render the last page, but I have a better, more sophisticated solution.

Add this piece of code as a bookmarklet, go to any Github repository and click it.

javascript:(b%3D>fetch(%27https://api.github.com/repos/%27%2Bb%5B1%5D%2B%27/commits%3Fsha%3D%27%2B(b%5B2%5D%7C%7C%27%27)).then(c%3D>Promise.all(%5Bc.headers.get(%27link%27),c.json()%5D)).then(c%3D>%7Bif(c%5B0%5D)%7Bvar d%3Dc%5B0%5D.split(%27,%27)%5B1%5D.split(%27%3B%27)%5B0%5D.slice(2,-1)%3Breturn fetch(d).then(e%3D>e.json())%7Dreturn c%5B1%5D%7D).then(c%3D>c.pop().html_url).then(c%3D>window.location%3Dc))(window.location.pathname.match(/%5C/(%5B%5E%5C/%5D%2B%5C/%5B%5E%5C/%5D%2B)(%3F:%5C/tree%5C/(%5B%5E%5C/%5D%2B))%3F/))%3B

I recommend bookmarking any site (such as this one) and then editing the bookmark by right-clicking it. I’ve set it up in a similar fashion.

I can’t for the life of me remember where I sourced this code, but if anyone has any leads, I’ll be sure to credit the author. :)

Xcode 🖥

I recommend downloading and installing Xcode at the start — it’ll take the longest. Once you’re done setting up, run the application once and download the required components. Also, for command line tools:

$ xcode-select --install

If you end up running the above command before Xcode installs you might have to run this command to switch to Xcode’s CLI:

$ sudo xcode-select --switch /Applications/Xcode.app

Other Applications 📲

I use MacApps.link (a flavor of Ninite — a Windows package management software) which handles the downloading of all other apps required on my work laptop.

$ curl -s 'https://api.macapps.link/en/chrome-dropbox-alfred-github-sublime-iterm-unarchiver-cyberduck-spotify-vlc-slack-whatsapp-discord' | sh

Choose whichever apps work for you.

IntelliJ IDEA 🧑🏽‍💻

I’ve been sucked into the IntelliJ ecosystem, and doubt I’ll ever be break away. It’s definitely worth the yearly Start by downloading the Jetbrains Toolbox, and through that get access to the latest version of IntelliJ IDEA Ultimate for Apple Silicon. It’s optimized for ARM chips.

For React Native dev of course I need to set up both native mobile dev environments. iOS was (unsurprisingly) straight-forward, but Android posed some problems. While the Studio initially installed without problems, when installing emulators I hit the real issues: the emulators that come with Android Studio don’t run on ARM Macs at all.

There is a ray of hope with some preview versions of Android emulator for Apple M1 coming from Google. But don’t get your hopes up too high as there’s already a list of known issues (no webviews and no sound being probably the biggest dealbreakers for me) and no activity in the repo in the past month and a half. I did manage to use it to run the app I’m currently working on, though.

Personally, I’m sticking with running my Android apps on a real device. Less hassle and faster, more reliable testing. It’s also been my workflow before switching to ARM, so that’s not much of a problem for me.

Chrome Extensions 🧭

• Ghostery | Google Analytics Opt-out Add-on | uBlock Origin | React Developer Tools

Brew ☕️

Homebrew, the essential package manager you need, finally officially supported Apple Silicon on the 5th of February, 2021. Preferably follow the instructions on their website, but for posterity, I’ve added the commands below.

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

..

$ echo 'eval $(/opt/homebrew/bin/brew shellenv)' >> /Users/{YOUR_NAME_HERE}/.zprofile
$ eval $(/opt/homebrew/bin/brew shellenv)

$ brew upgrade && brew update && brew install node && brew install yarn && brew git && brew install python@3.9

Rosetta 🌹

By now, if you’ve already installed a handful of apps, you might have already come across Rosetta. For now, you’ll have to make do with this translation layer until everyone switches over.

$ /usr/sbin/softwareupdate --install-rosetta --agree-to-license

cd into your ios/ folder and run these commands

$ arch -x86_64 sudo gem install cocoapods
$ arch -x86_64 pod install --repo-update

I also recommend duplicating iTerm and appending with _rosetta and running that application with the option ‘Open using Rosetta’

Mac AppStore 💻

When you first open it, there are a bunch of agreements you’ll have to accept. The only applications I care for are Helium, ColorSlurp, and Amphetamine.

I also downloaded

• Linear — issue tracking tool,

• Figma — collaborative UI design tool, and

• VEEER — window manager for MacOS

from their respective websites.

Troubleshooting ⚠️

If you recently got the computer and ended up inadvertently downloaded the software update for MacOS, I recommend restarting your computer at-least once. This solved some of the issues.

1. CocoaPods LoadError:

$ sudo arch -x86_64 gem install ffi
$ arch -x86_64 pod install

Or

$ brew install libffi
$ ln -s $(brew --cellar libffi)/*/include/ffi.h /usr/local/include/ffi.h

2. Missing architecture:

Inside BOTH Pods and your workspace, navigate to Build Settings -> Excluded Architectures -> Debug and add arm64. Remember to make this change in both places otherwise it won’t take effect.

Another reference from StackOverflow.

3. Class AMSupportURLConnectionDelegate is implemented in both ?? (0x1eb0e27a0) and ?? (0x1296d02b8). One of the two will be used. Which one is undefined.

Still figuring this one out. Fortunately it doesn’t affect development. Here’s another reference.

4. CompileC Error in ios/Pods/Flipper/xplat/Flipper.

In Podfile, update this line

use_flipper!

With this:

use_flipper!({ 'Flipper-Folly' => '2.3.0' })

Make sure to remove Podfile.lock and then run pod install after making this change.

Miscellaneous ⚒

Switching off last login time in terminals

$ touch .hushlogin

Generating SSH keys for signing commits/ pulling repositories

$ ssh-keygen -t ed25519 -C "viren@company.com"

Cleanse 🗑

Clean your build in Xcode. ⌘ + Shift + K

Purging the system might help.

$ gem uninstall ffi
$ gem list --local --no-version | grep cocoapods | xargs gem uninstall
$ rm -rf ~/.cocoapods
$ gem list

Restart terminal after checking the option ‘Open Using Rosetta’

$ sudo xcode-select --switch /Applications/Xcode.app

Then try

$ sudo gem install cocoapods
$ sudo gem install ffi
$ pod install

Whenever I run into an issue, I

$ rm -rf node_modules && cd ios && rm -rf Pods && rm -rf Podfile.lock

If nothing works, purge the entire repository and start over.

$ git clone git@github.com:{YOUR_REPO_HERE}
$ cd {YOUR_REPO_HERE}
$ yarn
$ cd ios
$ pod install
$ cd .. && yarn run ios-native

More troubleshooting hotfixes — CompileC AppDelegate.m Error.

I was recently assigned a technical project — to create a micro-service architecture and design the system from the ground up. After clearing the proposal and receiving approval from management, I thought it would be a good idea to post the results and keep adding to this document to further build on the insights I’ve gathered to improve the application. I love open-source solutions and Product Managers might also find this useful! Below are my findings.

Overview

My understanding from the prompt is that a user is looking for a matchmaking service (in the form of a native application) that can connect people to different business owners based on ​goals, interests, and industries.​ I thought it would be best to take a step back and approach this from a high level by first designing the Entity-Relationship Diagram (ERD). I’ve linked a preliminary UserAccountClass here as a ​GitHub Gist​ and also attached an image below:

While designing the architecture, I primarily focused on the Objectives field, which contained what the user was looking to get from the app.

Are you looking for a job, mentorship, to mentor someone, or exchange industry knowledge, or other?

Building a user recommendation algorithm should start with the user, which is why I wanted to start scaffolding the UserProfile first to get an understanding of how everything might work and add additional fields based on needs and requirements.

Based on the supplied mockups and interface, I divided Objectives into these four categories: Coffee, Collab, Tutoring, and Jobs. This would form the bulk of the ​RecommendationEngine​.

FRONTEND

I first needed to define the use cases and the most optimal infrastructure for the mobile application. Since I have previous experience in JavaScript, I thought it would be best to go with React Native (TypeScript) with a mix of firebase functions and hopefully a serverless backend. More on that later.

Using React Native (or Expo at the start) will achieve feature-parity across all mobile systems (iOS/ Android), limit the need for maintaining two codebases, while also having the added advantage of allowing for a future launch of the platform on PCs via React Native for Web.

For Market Research I looked at the onboarding process of Tinder:

1. They have their own login system and OAuth from several social media platforms. They also request your mobile number and the subsequent OTP for verification. Firebase has similar Authentication features, including one-time password phone verification.

I’d start by porting and converting over all the UI/ UX designs via Expo for quick iterations and rapid development, and then eject if and when native modules are required.

I’d use a combination of global classes/ themes while also introducing a highly componentized design to reduce redundancy. Each screen will be divided into separate components for reusability and modularity -- i.e, PrimaryButtonComponent for Call to Actions (CTAs) usable everywhere, UserCardComponents for use on the NetworkScreen and ChatItemComponent on the MessagesScreen. The Business Logic of the application (including but not limited to any calculations) will be completely separated from the designs and screens to allow for the future web launch.

The ChatScreen for each thread can be broken into the ChatBubble, which will grab the entire conversation and spread the props based on the senderUid (whether it’s a sent/ received message.) The Disconnect option in chat will purge the database of the entire conversation thread.

I’m not familiar with the Remarks option and how that might work but I’m imagining a pop-up modal that will allow a user to give feedback for a particular user. This can also be highly component-based. I’ve gone into more depth on the Report function in the Content Moderation part under the POTENTIAL FUTURE REQUIREMENTS section below.

1. Fuse.js can assist with the SearchBarPillComponent in the MessagesScreen and as defined by the UserAccount it can either be filtered on a name, full-text conversation or Objectives basis.

2. React Navigation to navigate the screens and Redux for State Management.

3. Updates can be sent over the air with CodePush and will allow for immediate bug fixes based on any errors reported by Sentry or AppsFlyer.

4. Dynamic User-related Notifications can also be pushed to the device with AppsFlyer.

5. Firebase Analytics/ Segment is a great way to visualize conversions and analyze user data.

6. I’d also use Hermes (for Android) to improve start-up time, decrease memory usage, and result in a smaller app size package.

7. For optimizations, you can either only add .webP images in-app or introduce a middleman bundler like Webpack to minify code pre-build. Heavy caching will be implemented (memoization) to reduce network overhead and unnecessary calls to the servers.

8. Internationalization via react-native-localize might help in a) providing translators a separate file to convert all text into whatever preferred language, and b) decoupling the business logic and screens from the actual generated text. Also, the more languages, the more global the app can become.

9. Testing would be a combination of Jest/ React Native Testing Library/ Detox for components + E2E unit tests

BACKEND SERVICES

The Backend would have multiple Services and I’ve attempted to utilize a micro-service architecture to a) decouple any authentication mechanisms (regarding access) so there aren’t any clashes and b) create a single source of truth so the user can always go back to the previous state if they delete the application from their phone.

The three journeys would be something similar to this:

1. Client (New User) -> ​UserCreationService​ -> dB

2. Client -> Web Socket -> ​RecommendationEngine​ -> sends the information on what Objectives a user requires (sends multiple micro shards, and makes a parallel call to each of these shards, and receives the stack of User Profiles for that particular day.)

3. Client -> Web Socket -> ​MatchingService ​-> ​ChatService

Each client request will be sent to a gateway and ProfileService to ensure the right person has access to the correct data. Every request will be preceded by a check on their access control.

Similar to how Monet, another newly formed dating app works, at the start we can match users globally and aren’t limited by country. ​This will form the basis for database optimizations on not just geolocational sharding but sharding by Objectives instead.

I’ve highlighted the necessary services and gone into more depth below.

RecommendationEngine:

1. Instead of Gender/ Age/ Location (which is what dating applications primarily use to form a recommendation), we can use Objectives, Job Title, and Location.

2. Pulls out all relevant people from the ProfileService, or it could just be storing the userIDs and the Objectives.

3. To optimize for latency, we can shard the data by Objectives, and later on by geographical location, once enough users have onboarded.

4. To strike a perfect balance between low latency and cost $$$, one can host a single virtual server for every shard per cell. A cell can start as high-level categories taken from various freelancing websites (such as Photographer/ Graphic Designer/ Filmmaker, etc), and if and when required we can create subcategories to reduce latency. For example, there might be a huge number of Photographers, but a dearth of magicians, which means you’ll have to break up the number of photographers but can include all magicians in a single shard.

5. The shards can also be balanced based on the Unique User Count (in a particular location), active user count, and # of queries from a certain region.

6. By employing the use of a TagManager/ TagService we can modify the type of recommendations a user receives and make the system modular. TagManager in this context would be the parameters you choose. For now, we can stick to Objectives.

RecommendationAlgorithm​ (part of the RecommendationService):

1. Active Usage: upon app open, updated lastActive field on UserProfile.

2. Collect Tags: Location, Objectives, Job Title, Location, Experience, etc

3. Grouping Users: In its preliminary stage, we can start by matching users to their needs and requirements based on the provided Objectives, and as soon as we have sufficient data and the value-add of a particular user we can start creating groups of users. This essentially means the combination of people who like your profile as a whole, and the people you like who like you back. This will pave the way for an ELO matching system and will generate better recommendations for new users. An example of this would be graphic designers matching with like-minded graphic designers for the pure objective of collaboration.

4. Pickiness/ bad actors: Low (or egregiously high) swipe percentage will affect the quality of your matches and reduce your ELO.

5. Reply Rate: penalizes accounts that have a low reply rate percentage

6. Progressive Taxation: A high-value account (for instance a mentee with 10+ years of experience in the banking sector) will get taxed and normalized to make the playing field more ‘fair’ for other user profiles.

7. GROUP BY Stacks: and if a user swipes left to a particular stack show the other surrounding stack cells. For example, a user who is employed as a photographer looking for collaborations might also want to collaborate with Graphic Designers (instead of just photographers) since there is some overlap.

UserManagement/ ProfileService:

1. Each user will have its own record and any creations/ updates/ deletions will be managed by this service. This will include Authentication of each request and the creation of a token for SessionManagement.

2. There can be multiple authentication mechanisms, for example, an EmailService, but for the sake of brevity, we can use Firebase.

3. Firebase can provide the relevant access controls and assist in authenticating requests. In the backend, we can also do some redundancy checks. A JWT token should work.

SessionManagement​:

1. To check and see if a user can interact with another user and that they’re active. Also links back to the RecommendationService since this serves as an important matching factor.

GatewayService:

1. Takes the request and asks the above service if the user is authenticated, and if resolves as true, forwards the request to the appropriate service.

BillingService:

1. Billing can be decoupled from UserManagement and for accepting different currencies since the application will be global. A neat way to remove this dependency would be In-App Purchases (IAP) but both Apple and Android charge hefty service fees, and sending users to a website to pay would be a better alternative option.

MessageService:

1. Peer Protocol. XMPP can work here. Web Socket (Firebase) or TCP (building out a protocol of your own).

MatchingService:

1. A table that lists the users who have matched with whichever other users.

2. If a user A right swipes, both the IDs are sent to the MatchingService and waits for the other user B to either swipe right or left to complete the relationship and allow for messaging.

3. If user A left swipes on user B, user B’s profile is marked as hidden to the user and not shown in subsequent stacks.

4. Every time a swipe occurs, the MatchingService table will check to see whether it’s a match or not. If it’s a match, the UID pair is purged from the database and marked as hidden for x amount of time for the user who swiped left. If the user right swipes the dB is checked to see if there are any existing pairs for both UIDs. If yes, they match, if no, a new record is created, and the profile is displayed in the awaiting profiles stack.

5. Note: If a user uninstalls an application, the only information you’ll end up losing is the number of swipes a user has performed, whether they have liked or disliked a profile. They’re going to get recommendations of a disliked user.

NotificationService/ ChatManagement:

1. Web Sockets/ TCP to keep the connection open and push notifications to inform when a user receives a message, a message is undelivered, and the order and timestamp of each message in a particular thread.

ImageService/ ImageHandler:

1. Users can share images/ videos/ and other assets in chat.

2. File Systems are static so you can easily build a CDN over this. Allows fast access and with aggressive caching the overhead and network requests will be negligible.

3. Cheaper compared to storing Blobs/ Distributed File System (DFS).

4. I would prefer to decouple the Images from the UserProfile and create an entirely different database that can, in the future, be used for machine learning and improved recommendations, or object detection. DFS + database with all the information for the particular images.

Monitoring​: Prometheus for server performance, latency, uptime, and logging/ Sentry for in-app error reporting.

DATABASE

Based on the services required, these are some of the requirements from the database:

1. Low Latency: Fast Search of Profiles.

2. Doesn’t necessarily need to be real-time: a new user doesn’t instantly have to be added to the RecommendationService but can be shown other profiles already in the database.

3. Easy to Shard/ Distributed: for scalability

4. Full-Text Search Support: there are a lot of parameters to search for (the tags) that need to be fully supported

5. HTTP Interface: TCP/ Web Socket

6. Structured Date: xml/ json

ElasticSearch accomplishes all of these requirements.

Client -> ​elasticsearch​ Feeder -> Kafka or any Queue (index all the data asynchronously) ->

elasticsearch workers (which will request the ​TagService​ and what shard you need to write into)

Firebase/ Firestore can also achieve all of these and have inbuilt helper functions. For a real-world setup, you’ll want to configure security rules for Firestore. Cloud Firestore favors a denormalized data structure, so it’s okay to include senderId and senderName for each MessageItem. A denormalized data structure means you’ll duplicate a lot of data, but the upside is faster data retrieval. There are a lot of tradeoffs. From the mockup, I could see a user can add images and content to chat also. Rather than storing the image data directly with the message, you’ll use Firebase Storage, which is better suited to storing large files like audio, video, or images.

If you’d like to go a more homegrown route, NoSQL Databases like Cassandra, which is good at querying for these kinds of data types, you just replicate the data, and depending on the query you build an efficient table. Distributed Database + AmazonDynamoDB work here too.

Also, Redis — using transactions to make this process consistent.

If you’d like more control you can use a relational database. The same concepts apply, you’d just have to employ Sharding.

TECHNICAL DEBTS

Working on a project of this breadth and also safeguarding against any single point of failure (leveraging the Master-Slave architecture), will introduce more challenges and complexity. The project will need to be thoroughly tested to ensure feature-parity as well as cohesion between all systems, whether that is client-machine or machine-machine.

I can also imagine combining the older, more mature freelancing platform users with this new app will be somewhat of a challenge and introduce a lot of bandaid code which should be ironed out before launch.

Fortunately, using the error reporting tools both on the client and the server-side should potentially solve both of these issues.

ROADMAP

Starting off by building an MVP of the application would be the first potential step. Knocking out the screens and while also figuring out how everything works together (business logic) might be cumbersome but well worth it at the start.

A web launch can be a possible future addition to the roadmap.

OTHER THOUGHTS AND POSSIBLE FEATURES

At the start, the number of users using the app will be fairly limited, and the best way to sidestep this and induce growth is to optionally include all accounts on the pre-existing platform and send them an email if and when they get matched by our users. This viable route will leverage the existing 30,000+ freelancers and generate leads.

I received a generous amount of help from Tinder’s unofficial API to get some sort of sense of what the required fields would be for a profile of this sort. I also went through the various freelancing websites to grab the CategContent enum to help divide and subsequently subdivide groups of users for lower latency during recommendations.

A move away from the Tinder landscape — if a user swipes too fast on a match notify them that they’re only receiving x number of matches a day in a particular stack and they should check out the profile properly for a) finding a hidden gem or something they can connect with the match, and b) improving the recommendation engine. Or, a counter at the top which decrements after every swipe.

POTENTIAL FUTURE REQUIREMENTS/ ADDITIONS

Content Moderation, whether people are using false images (celebrity pictures, copied pictures, no pictures at all) can be achieved with machine learning algorithms in ImageService. Text/ Chat Moderation, keeping the platform clean would be of paramount importance, and employing the use of a database that allows for full-text search would be a great addition to improving the community.

Feb 01, 2021 Update:

Imagine getting called out a few days after you share your thoughts:

Most startups (and big companies) don’t need the tech stack they have.

The overhead in setting up Kafka versus a very hefty Postgres (or BigQuery, or your relational database of choice instance hooked up to a web service receiving JSON sensor data) is enormous, because distributed systems are hard - really hard - and much, much harder than traditional systems.

Overarchitecture is real. As Nemil says in this fantastic post,

‘Early in your career, a surprising number of poorly engineered software systems are due to mistakes with engineering media.

In college and at bootcamps, your primary exposure to engineering is through engineering media like Hacker News, meetups, conferences, Free Code Camp, and Hacker Noon. Technology that is widely discussed there — say micro services or a frontend framework or the blockchain — then unnecessarily shows up in your technology stack.’

I’ll keep revising my architecture and stack to strike a better balance between convenience and sophistication.

I should probably focus on Cracking the Coding Interview. Instead, I came across this tweet — spurring the idea of detailing my approach to determining a user’s credentials via various means, whether that is through FLIR cameras, general stupidity, wordlists, or OSINT.
Disclaimer: I do not support penetration testing into national systems or any systems for that matter without authorized permission. This was purely an academic exercise.

Hint: focus on the Twitter username.

Extracting ATM pins w/ Infrared Cameras

Forward-looking infrared cameras aren’t particularly new — they’ve been around for ages, conveniently classified for military appliance until the technology was deemed appropriate for general use. Then also, FLIR only touched retail in the past four decades, first focusing on more profitable (and frankly more suitable) endeavors: settling in the niche B2B services industry of electrical and pipeline maintenance, outdoor adventures, and emergency rescues. Fast forward to 2014, industry leader and aptly-namesaked FLIR Systems launched their first consumer-focused mountable attachment — as a mobile case. Skimming their technical specifications, one can see the relevant Imaging & Optical Palette they use to color-code their images.

Gray (white hot), Hottest, Coldest, Iron, Rainbow, Rainbow HC, Arctic, Lava and Wheel

A picture here might be worth more than the above dozen colorful words:

Trust me, I love the resurgence of this technology. On my very own platforms, I’ve implemented Hotjar, which analyzes site traffic to generate heatmaps and recordings.

Armed with Google Analytics, I can, with a ~80% confidence, identify who is visiting our websites according to their frequency, activity, and location — even with anonymized data

And this entire flow is automated, neatly packaged, and shipped to my inbox week after week for me to glean over. If a lowly web developer can wield this kind of power, imagine what governments (NSA) and corporations (Palantir) are capable of achieving. But what if you’re looking for a more professional (or more nefarious) solution?

In Heat of the Moment: Characterizing the Efficacy of Thermal Camera-Based Attacks, a handful of UC San Diego researchers could theoretically guess the right pin code (typed into a keypad) 80% of the time if the image was taken immediately after. If the image was taken a minute later, they had a fifty percent chance of guessing correctly.

Another unnerving aspect of this potential vulnerability is as the heat signature fades with time, one can figure out the order in which the keys were pressed, with the dimmest (coolest) button pressed first and the warmest (hottest) pressed last. With relatively fewer numbers of permutations and combinations and a bit of solid guesswork, you can accurately predict and gain access. For safety deposit boxes use the opposite element — perspiration i.e. water or sweat — and reverse the methodology.

Should you now start removing rubber and plastic buttons, replacing them with metal keys? Are we always going to be defenseless to high-functioning sociopaths who measure our body temperature to see if we are lying about our unprofessed love?

TL;DR? You’re fine. Regular image sensors only detect near-infrared and not the mid to far IR radiated by hot objects. I’ve run ‘tests’ on DSLRs with the filter removed and wasn’t able to remotely capture the same level of heat on plastic buttons as the paper.

Or you can just follow my handy guide to safeguarding the (hopefully less than) ten bands in your checkings account:

• After use, an attacker requires physical access to the device in less than a minute for a higher success rate. Cherish the card machine for a minute longer. Maybe kiss it for good luck, but remember to be COVID-friendly though.

• This attack vector won’t work on all keypads — metal keypads reflect IR like a mirror, are highly thermally conductive, and dissipate heat quickly, which doesn’t allow for a thermal signature to be left behind.

• One can easily thwart attempts to extract the passcode by resting their figures on the remaining keys. This simple precaution produces a meaningless thermal signature. This isn’t even particularly hard, just smash the keypad a few times, or pass the bill to your inebriated friend right after.

Stupidity — The U1timat3 Anaphrodisiac

According to DataGenetics, 10.7% of the population chose 1234 as their ATM pin code. Ignoring this egregious error, there are a number of ways a person is vulnerable to credential theft, besides one’s own foolishness.

Statistically, one third of all codes can be guessed by trying just 61 distinct combinations

If you’re looking to choose the world’s least common pin then you could potentially go with 8068, which is statistically the least likely pin code (based on frequency). But that might have some unintended consequences, for instance, the least popular four-digit sequence might end up bubbling up to the top, like some sick twisted game of Nash Equilibrium, or for more gamer-inclined minds, akin to Gandhi in Civilization 1:

Allegedly, there was a bug that caused Mahatma Gandhi’s aggression to go to the maximum value of 255 due to an integer underflow error. In short, his aggressiveness stat is low enough that, upon adopting the democracy option and incurring the government’s inherent drop in militancy (-2), his default one point of aggression would sum to a negative value. This would cause it to wrap around to the highest value and subsequently turn Gandhi into a genocidal maniac. I’ve solved the complex math:

1 + (-2) = -1

‘Its fair to say that Gandhi could, on occasion, seem a little unnecessarily zealous,’ Sid Meier, main developer and designer of the turn-based strategy video game, concludes.

The existence of this bug was denied by Meier, and has no mention on the internet prior to the fifth installment of the award-winning game, Civ V, where Gandhi has an unnervingly high nuke rating (possibly as a homage.)

Assumptions based on data and empirical evidence can backfire and instead make one shoot themselves in the foot. Don’t be a sheep. There is always a relevant xkcd:

I am a sheep. I shot myself in the foot. Enter Security.org to call me out:

In a paper by the Computer Laboratory team at the University of Cambridge, they utilized a regression model to identify the factors which influence consumers picking a particular pin code combination. I’ll append parts of the abstract for the click-lazy:

We find that guessing PINs based on the victims’ birthday, which nearly all users carry documentation of, will enable a competent thief to gain use of an ATM card once for every 11–18 stolen wallets, depending on whether banks prohibit weak PINs such as 1234. The lesson for cardholders is to never use one’s date of birth as a PIN. The lesson for card-issuing banks is to implement a denied PIN list, which several large banks still fail to do. However, blacklists cannot effectively mitigate guessing given a known birth date, suggesting banks should move away from customer-chosen banking PINs in the long term.

People are stupid. Or Naive. I prefer to harbor on the latter for the sake of humanity. We choose recurring passwords for consistency (2244), patterns that can be etched into memory (2864 - north, south, east, west), or important dates (birthdays, most likely mothers). How hard would it be for an acquaintance to ask about your mother’s birthday? Or know about your affinity for compasses? Or slight OCD tendencies?

Know your customer. Their likes, dislikes, their innermost desires, and some sequence of numbers that might make them exploitable.

According to our computed model, the following blacklist of 100 PINs is optimal: 0000, 0101–0103, 0110, 0111, 0123, 0202, 0303, 0404, 0505, 0606, 0707, 0808, 0909, 1010, 1101–1103, 1110–1112, 1123, 1201–1203, 1210–1212, 1234, 1956– 2015, 2222, 2229, 2580, 3333, 4444, 5252, 5683, 6666, 7465, 7667.

Remember to swap these out a year down the line.

Enough numbers, let’s move onto characters. In 2015, security researcher Mark Burnett released a list of 10 million passwords. The data consisted of

• Portions of most major data leaks in the last five years where there were plaintext passwords leaked or published later

• Thousands of smaller leaks found on Pastebin and similar text-storage sites

• Google Searches/ Alerts for leaked databases

• Forums that share premium file-sharing and gaming accounts + porn passwords

• Dumps discovered through torrent searches

There is a defunct subreddit dedicated to sharing research on the list. Some of the most popular posts performed Levenshtein distance between usernames and passwords, checked patterns/ character counts, and listed potential flaws of the set.

Slight digression: 69 was the 3rd most used pair-combination. A commenter replied:

I assume because most users were born in 1969?

This reminds me of when my mother wished her neighbor on April 20th because the Wi-Fi network name had 420 appended at the end of his SSID. Another one:

Unrelated, but did anyone else notice that the number 5683 was one of the top 10 most common PINs? I can’t think of a single reason why that would be.

It spells ‘LOVE’ using the alphanumeric code on the keypad.

The patterns of numbers in passwords offer a brief glimpse into human psychology, similar to the insights gleaned from the pin code paper. Birth years are a common trope, as well as repetitions (111/ 999). 768 is an important number for Muslims.

Another insightful comment from Hacker News:

When sites require a digit, everybody appends ‘1’ to their usual password. The exponential declining frequency of subsequent digits is because when passwords ‘expire’ folks just add 1. The short lifetime of site usage results in that decline. Just thinking out loud.

But the onus isn’t only on users. Strings like Password2020used to score high on password strength meters because it fulfilled the abjectly bad base criteria being:

• Longer than 8 characters, and

• One of each small letter, capital letter, digit, or symbol.

Take the Password Test. If you’re curious about whether your password or email has been compromised check HaveIBeenPwned. The website doesn’t store any passwords — their Privacy Policy has a brief on how they accomplish this. Technicals for nerds.

HSBC forces you to use a six-digit pin code compared to the customary four digits, increasing the number of combinations hundred-fold (10,000 -> 1MM). But they also renege on other responsibilities — like providing an appropriate login user interface.

And God save you if a bank limits the max length or bans the use of special characters.

TL;DR? Rotate passwords regularly, definitely don’t reuse them, and forget your mother’s birthday for the sake of your bank balance. And don’t use xkcd’s correcthorsebatterystaple. Do we really need to worry though? Not really.

1. The dataset is not representative of all passwords. The list also has no indication of any source or user attitude. Awful websites with awful security will store awful passwords. If one would go on a flower companies website they wouldn’t care to use a strong password, especially if it’s a one-off visit. Their bank credentials and social media accounts will most likely have a more well-thought-out password.

2. password is #2 on the list only because frequency can’t possibly decrease. In retrospect, it might be beneficial to have a list of common passwords that people don’t end up using — one’s that crackers keep regurgitating when attempting to access your account. Your unique passwords could theoretically never reach the top of the list because they’re unique. Plus, the dataset is incorrigible:

Many dumps include passwords in a hashed format that requires you to crack them yourself.

3. Use LastPass or any other password manager to securely store and generate high-entropy strings. My hot take is you shouldn’t even remember your credentials. The less human intervention the better. People introduce errors.

4. Research suggests that users’ first option for the language chosen for their password is their mother tongue. Add some french in there. I can help.

What can companies do on their side?

1. Like the Cambridge Team suggested, companies can take the leaked dataset, run the uniq command to filter out duplicates, and create a blacklist to deter people from choosing the easiest path. That’ll easily make the list redundant.

2. User-accessible features that assist in pasting passwords from password managers instead of creating one’s own convoluted login system. HSBC, please.

Big Black Box

Let’s say you’re tech-savvy enough that you follow all the above precautions. You’re emailing confidential information and have redacted the necessary fields. You’re still not technically safe. There have been many advancements in software tools that make it surprisingly easy to recover text from pixelated images or PDF files.

Not only can the length of the redaction give an estimate on the password length, but for variable width fonts, one could even rule out many passwords using pixel measurements between the text on either side.

It took a while to convince people to not use a bit of Gaussian Blur because it’s extremely insecure. Well, get ready for round 2. Case in point: the child abuser who was caught by simply reversing a Photoshop filter.

Police [essentially 4Chan] took a photo with a ‘swirl’ effect of the paedophiles face and reversed it to reveal a very usable picture. So good in fact he was found and arrested.

One can ostensibly reverse-engineer the underlying blur encoder and in turn, decode any screenshot. It’s a cat-and-mouse chase × arms-race. Why don’t people use black boxes properly? Simple: it attracts more attention. A redacted rectangle with sharp edges has much more contrast than all the other elements in the image, becomes visually dominant, taking the front row seat.

A pixelated area communicates more clearly that there’s information present that is hidden from you- but not from others. It also can’t be mistaken for a design element.

In 2017, two French ‘hackers’ reconstructed a blurred-out cryptocurrency QR code on TV and claimed bitcoin cash worth $1,000 (or £760 at the time of discovery).

Sassano and Storck, [the hackers], have explained the process in detail in a blog.

As always, you gotta share the content for those clicks and page views. Cough.

One can even decensor Hentai with Deep Neural Networks. A project humorously titled DeepCreamPy handles the dirty work so you can take care of your own business. Remember the markup tool debacle in iPhones? It was actually mostly transparent and barely hid any information, and ended up becoming a meme to slightly hide people’s names but in actuality dox them. You can see many instances of this on the /r/Tinder subreddit. The funniest part is it’s the most commonly used pen tool because Apple purposely choosing to place it front-and-center in their user interface, and giving it the widest stroke. People can’t be bothered, and end up picking the easy option. Image metadata (if not properly stripped) can geolocate you to a very accurate degree.

So that begs the question, can you do the same with faces? Sure the contrast is a lot worse but the added frames from the video might serve as a sort of interpolation? In some cases even blurring faces might be a bad idea. Just because we are unable to unblur a face today doesn’t mean we are unable in 10 or 100 years. Food for thought.

TL;DR? Be careful with everything you put up online. The age-old adage rings true:

what you put on the internet stays there forever, etched/ carved in stone.

1. I pipe all of the images I upload online through ImageOptim first.

2. Use the Adobe Acrobat Redaction tool. It’ll obliterate everything + the metadata.

3. Image formats like JPEG have a thumbnail (part of the EXIF metadata) stored in them, which may not be updated when you edit the image. Also, leave generous margins. JPEGs can have compression artifacts that leak information outside the boundaries of the object. I’m sure you can sense my hatred towards this format.

4. Are you printing a document? Follow these instructions to safeguard yourself —

   Redact -> print -> scan -> distribute.

   US courts and lawyers are finally starting to learn. Or for the paperless moguls:

   Redact -> convert to image -> convert to PDF -> distribute.

And companies?

1. In Photoshop, the Filter > Pixelate > Mosaic option should have a checkbox called Secure? or Security Noise. Or there should be a separate filter called Pixelate > Redact. Ideally, it would use some intelligence to figure out the size of characters/ symbols in the selected area and automatically figure out the right combination of pixel size and noise.

2. Or, Keep It Simple Stupid (KISS): flood the whole rectangle with black. If that’s the only layer of the picture, there’s no need to worry if the ‘secure noise’ is secure enough, or if it keeps staying secure two years down the line.

3. Pixelization algorithms should implement some degree of brightness and chromatic random noise in order to defeat this new wave of attacks.

Might be overkill.

Open-source intelligence (OSINT)

Won’t share too much here because I might lose access to this tool, but there are a number of completely legal websites out there that one can use to perform KYC. The paid solutions provide a trove of data to:

1. locate persons of interest

2. Uncover associations between people, addresses, phones, and social handles 

3. Determine the credibility of sources, witnesses, or suspects

4. Track changes in historical online and offline identity information

5. Connect personal, professional, and social information

Think of the site as a search engine for people. Social media resources like Twitter and Facebook are especially useful, wherein you can programmatically access or publicly scrape information on target users. Not using API keys grants you an additional level of security as well as the possibility of bypassing developer limitations.

TL;DR? Keep a tight check on your social media presence and old content purge regularly, after saving it somewhere. The 3-2-1 backup rule might work well here.

Please Charge Your Phone with My Wire

Hey, we’re back to iPhone accessories. Last year, at the annual Def Con hacking conference, security researcher MG introduced an Apple charging cable capable of remotely connecting to a computer. Talking to Vice, the devilish creator suggested you may even give the malicious version as a gift to the target — the cables even come with some of the correct little pieces of packaging holding them together. MG said:

It’s like being able to sit at the victims computer but without actually being there.

The cable comes with various payloads, or scripts and commands that an attacker can run on the victim’s machine. A hacker can also remotely ‘kill’ the USB implant, hopefully hiding some evidence of its use or existence.

He is selling the cables for $200 each. Pricey, but Apple has always thought differently.

TL;DR? Don’t use random people’s wires, and if you have to, don’t ‘trust’ the wire.

Back to Basics (ABC’s)

Let’s bring everything we’ve learned today, together. Once you’ve figured out the specifics of a user, all you really need to crack their password is a wordlist along with the internet’s most popular cracking tool, John the Ripper. He supports four modes:

1. Single crack mode: Tries mangling usernames obtained from the GECOS field, and tries them as possible passwords

2. Wordlist mode: Tries all words in the provided wordlist (see above dataset)

3. Incremental mode (aka Brute-Force attack): Tries all possible character combinations.

4. External mode: Optional mode in which John may use program code to generate words. One can add their own rules to drastically decrease solve time. What rules you might ask? All the exploitable details about the target user you collected (including, but not limited to, their parent’s/ pets name, birth year, school, etc)

Now, let’s create a hypothetical situation:

Using a wordlist dictionary (sourced from the internet) that has approximately 1,493,677,782 words (sized at 15GB) of the most commonly used phrases and database leak of existing user passwords, one would assume that it would take forever to crack the password and gain access. Combine this with external mode, and you’re exponentially multiplying the number of words. You’re safe right? Nope.

According to JtR benchmarks, an AWS c5a.24xlarge instance can perform ~408 billion checks per second (is it cracks? not entirely sure). Even if you multiply your standard wordlist by 100,000 it would take a little over 36 seconds* to run through the entire set. Spread and parallelize that across multiple instances and you have a formidable password cracking tool. Now, not everyone has access to a beefy c5a, which costs ~$3.696 per hour in the cloud. What about running the cracking tool on your own laptop? You’ll likely easily reach 5MM c/s on a single core, which isn’t too bad. On an 8-core machine, you’ll end up taking 43 days. Probably best to fork over that cash to our friendly neighborhood book rental company.

For the time-strapped but cash-heavy individuals, please run Jack on x1e.32xlarge and share the results. At 7 times the hourly cost of the archaic c5a, I can’t wait for an optimized program to cross 1 trillion c/s, but for someone else to front the dough.

* my math might be egregiously wrong here, I wrapped up this section at six in the morning.

Epilogue

So what can be done? Are we out of luck and is privacy dead? The famous (and somewhat obligatory) quote from Edward Snowden, the NSA whistleblower:

Arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don't care about free speech because you have nothing to say.

TL;DR? Follow the tl;dr’s and you should honestly be fine.

Further Reading

1. Google Wallet pin code can be cracked in as little as 18 minutes— Sept ‘11

2. Smartwatches Can Be Used to Spy on Your Card’s PIN Code — Jan ‘16

3. Is 123456 Really The Most Common Password? by security researcher Mark Burnett.

4. A Glimpse Into the World of Internet Password Dumps, also published by Burnett.

5. Analyzing the Patterns of Numbers in 10 Million Passwords by Max Woolf, insightful.

6. PasswordsCon — brilliant research papers presented here.

7. Unmasked by WPEngine. The psychological reasons a person chooses a password.

8. The Secret Lives of Numbers currently hosted at Turbulence. An exhaustive 2002 empirical study to determine the relative popularity of every integer between 0 and one million. The authors themselves pitch the project best:

   We surmise that our dataset is a numeric snapshot of the collective consciousness.

9. Recovering passwords from pixelized screenshots by Sipke Mellema, creator of Depix.

10. Why Blurring Sensitive Information is a Bad Idea by Dheera Venkatraman.

11. Refocus-it can be used to refocus images acquired by a defocused camera, blurred by Gaussian or motion blur, or any combination of these.

12. Why You Should Stop Using Other People’s iPhone Cables — Sep ‘2020

13. Hashcat — advanced password recovery. World’s fastest password cracker.

14. Privacy matters even if ‘you have nothing to hide’

Everything comes full circle. After attempting to reverse engineer a cracked version of Turmoil (a simulation game in which you drill for oil and expand on your capitalistic tendencies) earlier this year — I now start my investment foray into the hottest liquid commodity.

All information provided in this report is for informational purposes only and should not be deemed as investment advice or a recommendation to purchase or sell any specific security. One can safely assume the author holds a position in any discussed security. Their economic interest is subject to change without notice.

Two Sides of the Same Lubricated Coin

When it comes to physical oil, there are different grades. The two most popularly traded are:

1. Brent North Sea Crude (commonly known as Brent crude) and

2. West Texas Intermediate (WTI).

Brent refers to oil that is produced in the Brent oil fields and other sites in the North Sea.

I use $BNO to track Brent, $USL to track the latter, and $DBO to cover the entire liquid gold commodities market, which has a mixture* of mutual funds, United States Treasury Bills dated for the end of Feb + mid-March ‘21, Invesco’s CLTL, and ~200 basis points of USD Currency. DBO also provides exposure to light sweet crude oil (WTI), which is the most popular oil benchmark in the world. Now one might ask why I’m not adding $USO to my ETF/ ETN basket. Surely with one of the (comparatively) lower expense ratios (0.73% compared to the category average of 0.76) and largest assets ‘under management’ ($3.65BB), and the added advantage of decent liquidity if I wanted to sell at short notice, I’d jump on the proverbial cargo boat. The issue is the relatively higher price and along the same line contango. From the Investopedia page:

Contango is a situation where the futures price of a commodity is higher than the spot price. In all futures market scenarios, the futures prices will usually converge toward the spot prices as the contracts approach expiration. Contango tends to cause losses for investors in commodity ETFs that use futures contracts, but these losses can be avoided by buying ETFs that hold actual commodities.

I choose USL because as its name suggests, they diversify across multiple maturities and rebalance on expiry, potentially eliminating the adverse impacts of contango.

USO suffers from severe contango making it more appropriate for short-term traders.

I guess I’m growing up and turning into a ‘seasoned investor’. Also — why not invest directly in the futures market to bypass this nonsense? Nope. I’ve tried and failed (read: my brief stint in soybeans, culminating in a piqued interest and eventual pivot to insect feed farming). If I’m dipping my toes again in commodities I’ll only stick to exchange-traded funds (or notes). As the Invesco description rightly states:

This product may be a good choice for investors looking to gain exposure to futures contracts on fossil fuels, but do not want the risks associated with a futures-contracts.

I digress. Year-to-date, they’re all doing horribly. Across the board, they’ve lost ~30% of their value, with outlier USO taking a huge hit of 68.61%. But since the time I exposed myself to the oil & gas sector, which was about twelve weeks ago, month-on-month they’ve consistently gained 3-4%.

YTD Performance for select O&G ETFS, ranked by assets (in ascending order) --
USO: -68.61%
DBO: -22.96%
UNG: -42.94%
BNO: -39.86%
USL: -27.05%
UGA: -30.01%
OILK: -61.95%
DBE: -27.28%
OIL: -27.52%
UNL: -5.13%
GAZ: -39.29%
RJN: -46.51%
JJE: -42.95%

Data from etfdb, dated Dec 16, 2020

* I really wanted to use emulsion here instead.

Some Barely Passable Technical Analysis

Now the burning question — why am I switching gears in the final month of the final quarter of 2020 from my usual tech and biopharma plays to not only an entirely different sector but a totally unrelated industry? I believe oil is poised to make a huge comeback next year. This year, and the pandemic, has been particularly brutal on our beloved time-tested corporations and combined with a huge demand and a dramatic increase in Airbnb rentals and pickup in flights (especially on Thanksgiving), we’re looking at a complete overhaul for shale. Now, some cool graphs!

and

Now one might look at the actual dataset from TSA checkpoint travel numbers and point the number-of-passengers discrepancy, but why should I care? Sure, total traveler throughput has fallen 33% compared to last year, but flights are still being chartered. Also, the initial investment is literal pennies on the dollar after COVID-19 ravaged the market. Warren Buffet seems to agree with my acuity:

Buffett has a reputation for spending his money fairly parsimoniously. Meaning he buys good companies when they are cheap, and famously, rarely sells them. I think there are two things that he likes about the pipeline business. One, it's a tolling business with long-term take or pay-type contracts and without nominally much exposure to commodity prices, although as we have seen those chickens will eventually come home to roost. And, as WB has told us previously, he likes businesses where he can just collect fees.

Read: Why Warren Buffet Is Betting Big On Oil & Gas Pipeline Companies
Disclaimer: I wouldn’t pay too much heed to information from a website named oilprice.com but they do have some neat insights.

We’re slowly converging on the equilibrium point, akin to contango!

Let’s correlate random data together and compare our ETF/ ETNs 2-year charts to another prominent oil player of the S&P 500 (which recently closed around mid-40s — laughable after holding the #1 spot in 2008): Exxon, ticketed as XOM.

Correlation = Causation?

One would expect similarities between a company whose primary industry focus is Oil, but comments by current CEO Darren Woods suggest otherwise:

‘I believe the assumption that affordable energy and a cleaner environment are a zero-sum game is mistaken. It underestimates the power of technology. The zero-sum view is a static one, and the world of energy is anything but static. But technology changes the equation.  It makes a dream – growing the economy while reducing emissions – a reality.’

Read: ExxonMobil and Global Clean Energy Holdings sign agreement for renewable diesel — Aug ‘20

Interesting take from a Rockefeller company, especially when in 2015 they prefaced a shareholder meeting with the then CEO making fun out of clean energy by commenting ‘We Choose Not to Lose Money on Purpose’. He then went on to get nominated as Secretary of State by then-President-elect Donald Trump on December 14, 2016, almost retiring four months early and potentially giving up a $180MM retirement package. He did redeem himself later:

following Trump’s strangely political and inappropriate speech at the annual Boy Scout Jamboree, Tillerson, according to NBC, was so offended that he was going to resign, until Vice-President Pence, Defense Secretary James Mattis, and the incoming chief of staff, John Kelly, persuaded him to stay. That same month, Mattis reportedly attended a meeting of national-security officials at which Tillerson referred to Trump as a ‘fucking moron.’

Read: Rex Tillerson at the Breaking Point — Oct ‘17

And I don’t want only want to shame Exxon alone. BP, Shell, Eni, Chevron all have reneged on their idealistic green-thumbed promises:

Despite the growth in renewables, big oil only spent 1% of its combined budget on green energy schemes in 2018.

A 2019 report by Matthias J Pickl — an economics professor at King Fahd University of Petroleum and Minerals in Saudi Arabia — discussed whether oil companies are transforming themselves into energy firms.

Furthermore, growing concerns about climate change following the Paris Agreement may provide an additional drive for such strategy to hedge against hardening investor sentiment towards carbon emissions.

Playing Devil’s Advocate

Why is it so difficult to predict oil trends? Well, we’re living in tumultuous times.

Short-sellers (like me) have lost more this year on Tesla than ever before.

According to analysis by S3 Partners, short investors in Tesla— those who placed bets in the market that its shares would lose value — have lost $35 billion on those positions so far this year. To put that loss into context, the US airline industry posted combined net losses of $24.2 billion, excluding special items, through the first nine months of 2020, the worst losses the industry has ever reported.

The automobile company is also poised to join the S&P 500 with a 1.5% weighting (compared to Apple’s 5.8%) on Monday, December 21, 2020. And I would prefer not to comment on Musk’s meteoric rise from multi-billionaire to current Twitter meme-lord. In a span of 9 hours on December 14th, he shared these golden nuggets:

Will this one trade put me in the black for the next year? I’m not sure. These publicly listed companies have to deal with new regulations, comply with ESG principles, and face the highest possible level of government scrutiny before they stand a chance of turning a profit in ‘21. Oil supply is currently outcompeting demand, and we might never go back to the way it was before. And if you look at the 5-year-window of any O&G corp you see red lines and losses. It might already be too late — the gold rush is finally over. How am I hedging my bets? I’m being less risky, by investing in ETFs of commodities instead of the futures market, but also riskier, because of the incontrovertible proof that oil is a dangerous business and I’m probably way over my head. But combined with stop losses and my unnervingly high-risk profile (thanks to crypto), I do believe oil will make a good enough comeback to lock in profits. I would like to close on two questions:

• Will the R² of these big conglomerates stocks and oil ETFs continue along with these trends or show some meaningful divergence in the near future?

• And when?

I don’t believe they’re going to be changing teams anytime soon, but nevertheless, I’ve included more resources below for you to accurately judge and ponder for yourself:

• Exxon Quietly Researching Hundreds of Green Projects — Nov’ 17

• ExxonMobil’s Failure to Go Green Could Worsen Its Financial Future — Nov’ 20

I’ll leave this as an exercise to the reader on where the road less taken (and frankly hopefully less polluted) might take us. Also, please let me know so I can exit some positions.

Further Reading

1. Shell, Eni Settle Dispute With Kazakhstan in $1.3 Billion Deal — Dec ‘20

   The deal unlocks plans to boost output from the Karachaganak [oil] field.

Finding a reputable, trustworthy developer from India for a client project is challenging. Especially if you’re navigating this minefield alone. In this write-up, I try and make the process easier by identifying a few core issues and how I work around them. Please Find Attached: suicide rates in India (due to the rat race), mitigating ‘overpromising and under-delivering’, weeding out subpar developers through rigorous interviewing, and why I hate certain phrases.

Indian-based developers aren’t all that bad. Some of my most extensive projects have been outsourced to experienced IITians and they’ve handled deadlines with relative ease — with comprehensive code documentation, unit tests, and a close feedback loop. And they’ve been handsomely rewarded for their efforts. But for all of these software engineers, soft skills are something that lacks across the board. This problem rears its ugly head in many forms:

1. Overpromising and Under-delivering — setting proper expectations

2. Fake Credentials — spring cleaning

3. Email Etiquette — why I hate the phrases do the needful, P.F.A, kindly revert back.

I attempt to address each point of contention below.

Rat Race

For some reason, unbeknownst to me, all of these young graduates fall into this communication trap. They’re raised in extremely competitive environments combined with heavy tiger parenting, where if one loses a contract or can’t secure a job they’re deemed unworthy and fall behind. An unfortunate byproduct of this perceived failure is the uptick in average daily suicides in India (381), with a 3.4% increase in observed suicides in 2019 (139,123) as compared to 2018 (134,516) and 2017 (129,887), as reported by the National Crime Records Bureau.

In 2019, a person in India died by suicide every four minutes.¹

Now, not all deaths can be attributed to career/ educational aspirations, and the farmers of Sikkim and Karnataka are indicative of this causal relationship. Different reasons: droughts, low yield prices, exploitation by middlemen, and loan defaults lead them to cut their lives short. But still, the statistic holds true, because a kisaans inability to provide for his or her family is just another synonym of frustration

India is the greatest contributor to the absolute number of suicide deaths.
— Global Health Data Exchange

In a pathetic attempt to curb these numbers, The Indian Government decided to deem suicide illegal, whereby survivors would face a jail term of up to one year and fines under Section 309 of the Indian Penal Code. Trust me, this irony wasn’t lost, especially after Parliament decided to repeal the law in 2014, and in April 2017, decriminalize suicide by passing the Mental Healthcare Act. Baby steps.

¹ The word ‘committing’ implies performing a cardinal sin through a deliberate act, however victims usually do not feel as though they have a choice. And while etymologically speaking this wording might not hold a lot of weight, according to Dictionary.com, replacing the phrase with died by suicide is a small change that can have a big impact.

The Duality of Man and One’s Achievements

I’ve digressed a bit, and since it a month has passed since Mental Health Day, I’ll get back on topic. What I’m trying to say is communal pressure pushes these developers to carve unrealistic goals which unfortunately results in one either losing trust in their ability (or lack thereof) in performing the tasks at hand or dropping them entirely. I pay well above the market rate and expect solid results.

I understand their predicament, but can’t empathize.

The time wasted, vetting, hiring, and peer-programming are lost and can never be gained back.

Playing devil’s advocate, Melissa Rutherford provides more context:

People who over promise and under deliver are often thought of as lazy, self-indulgent and irresponsible, when the opposite may be just as true.

We may get bored after the exciting ‘promise’ has passed.  Then we may begin to fear failure and avoid or procrastinate the now boring task, until it is too late.  We may become distracted by more urgent or fun tasks, telling ourselves that we will do it right after we do this other thing.  This results in last minute urgency and a poor job, thus self-fulfilling that prophecy of failure – and round and round we go -ugh.

Another concern of mine is the lack of communication between outsourcer and subcontractor. I myself had lived through countless horror stories of following up with developers numerous times over the course of a single day so I can then report back to a client. The most common reason why this persists is a direct tangent to the aforementioned issue — when one is knee-deep in a project and a deadline is nearing, they drop the ball and under-communicate to ‘attempt’ to focus on a particular feature. I’m not immune: I’ve done this countless times and have been chastised by clients and employers alike. I go into a hole (my room) and then try digging myself out by rushing to build out a screen or functionality when in actuality I should have known my limitations and either asked for help or asked for more time to address the problem.

A better approach for an engineer would be to identify the project scope at the very start, define strict end goals, and have complete clarity from both channels. Take a few days off and revisit the contract so you know exactly what you’re getting into, and if you have one iota of doubt about your ability to complete the project, be clear about expectations vs. reality. The client will greatly appreciate truthfulness and transparency over promises, promises, promises.

Also: Buffers. Add ten more days for scope creep, and then a few more for bug fixes. You’ll thank me later.

The Ancient Eight Forged Diplomas

In my four years of outsourcing projects, I’ve rifled through hundreds of resumes and CVs — some patterns emerge. Credentials, educational qualifications, plagiarism are rampant and omnipresent when you’re doing work online. If you ever feel something is amiss, one can generally connect the dots with a quick search and a perusal of their online professional profiles.

I’ve haven’t yet hired a developer who doesn’t have an online presence or glowing recommendations.

GitHub repositories are indexed frequently, and committed code can easily be searched. I use a combination of Grep, GitHub Search (unfortunately still lacking), and concatenated Google operators to drill down and find instances of forks. You can also just have a quick call with them to walk you through their favorite project.

For coding interviews, I provide the developer a single prompt, wherein they can either tackle the simpler 1-hour project or take on the 2-hour extensive project, (which has a handful of more accessibility features, for instance, unit testing, documentation, etc.) Prospectives can choose either, I hold no prejudice, but I do set hard limits on the time they can take to solve the question. This kills two birds with one stone: I respect their time and don’t waste it, and I get to see their creativity flow.

Later, we get on a call and once they’ve presented their application I ask them how they would plan to future proof or extend/ add features. If they had one more day what would they do? What about a week? A year? Ten years?

This provides me great insight on how they would operate and the extent to which they would think to hash out a solution. Some interesting nuggets from Twitter below (which in hindsight might be too pretentious and asking for a lot from the Indian subcontinent, but the replies are worth checking out):

Jeff Bezos once said:

‘I’d rather interview 50 people and not hire anyone than hire the wrong person.’

Please take advantage of the Amazon Interview Question Bank:

• Tell us about a time when you missed a deadline or productivity target?

• Describe a time when you struggled to communicate something to your boss, colleague, or customer. How did you manage to get your message over?

• Can you describe your most difficult customer and how you were able to handle their needs?

• Can you tell me about a time when you were more than halfway through a project and had to pivot quickly due to an unexpected change? How did you handle it?

• If one of your close work colleagues stole a $1 item, what would you do?

Why I Prefer Canned Replies

Onto my pet peeve. You'll never find the phrases do the needful, please find attached or, my most hated kindly revert back, in any of my emails. For some odd reason, I detest when I read them in one of the back-and-forths from my subcontractors. I’m guessing, years back, those particularly worded emails burnt me and have now left a bad aftertaste in my mouth. It's not the phrases themselves, it's (almost always) the context they are used in. They signify babudom at its worst. Where did that saying come from?

Likely of British origin, during the Raj. I can imagine ‘Kindly do the needful, old chap’ scrawled in the margin of an official missive, as it languidly flitted from one administrative table to another. Many such anachronisms exist; and thankfully, they're dying out one by one. A few decades ago (in the last century, to be precise), it wasn’t very uncommon for official letters to be signed off with

I remain, Sir, your obedient servant.

Later, it was shortened to yours obediently and now has vanished altogether. But why are these phrases still ubiquitous in Indian-English-speak? It partly comes from the language the bureaucracy uses and expects in communication, and given several sociological factors like

• India’s socialist history combined with a

• weak pre-92 marketplace,

• and a strong presence of state-owned businesses (even today)

hereditary roots/ the way you’re raised

• learning from the grandparents and parents who worked in this bureaucracy

educational misnomers

• with schools failing to teach soft skills like written communication — which won't help you ace exams and get a job, or push you forward in the rat race, so, therefore, aren’t important — pushing templated answers instead

and company norms, business practices, and formal mannerisms

1. with workplace culture influencing one’s speech and also what their mentors and management previously did when they were junior (and how they set the tone) — somewhat akin to the polite language you hear in the U.S. South.

You can pick up habits easily when you are not sure what is the right way to behave.

The Indian white-collar mindset is already set in stone, and it’s proving difficult to enact any sort of change. Now, you might say this has no material impact on my life, I’m being facetious, and to some extent incredibly racist, but, for me, words matter.

It showcases a want to break away from the norm, explore possibilities by yourself, and most importantly, have your own voice. But forget all of this, I’m the one hiring and paying developers, and similar to Airbnb having their coding style guide, I can have my very own format where subcontractors should abide by and toe the line.

According to updated numbers released by the U.S. Census Bureau, 99.98% of all housing units and addresses nationwide were accounted for in the 2020 Census as of the end of self-response and field data collection operations on Oct. 15, 2020. Let’s leverage this data for common good. Here, I take advantage of frequently common surnames to gain free wifi access at hotels.

Many hotels ask you to provide your last name and room number to use their WiFi. According to the 2010 U.S. Census Bureau, around 3% (or ~8,863,537) of people have

Smith, Johnson, Williams, Brown, or Jones as surnames. // 2.87*

And this number is only increasing. Therefore, theoretically, in a hotel with 50 rooms, there is a 78% chance that one could sign in by just trying these names for each room. It would be trivial to write a script that accepts a range of numbers and a list of surnames and generate pairs of permutation of combinations for input.

In fact, when I tested this using Smith as a proxy, I got access by my ninth room.

YMMV, but I doubt this would be theoretically possible in India. Too many edge cases.

I’ve curated a list of common surnames in each country (sorted first by region, then alphabetically) that could potentially save you on incidental costs during travels:

Asia

Bangladesh (Bangla Romanization): Ahmed, Ali, Akhtar

China: Wong, Lee, Cheung, Chan

Japan: Sato, Suzuki

Korea: Kim, Lee, Park, Choi

Philippines: Santos, Reyes, Cruz

Turkey: Yilmaz, Kaya, Demir, Şahin // does the cedilla hook-tail in Şahin count?

Vietnam: Nguyen, Tran, Le, Pham, Phan // 70.1%

Also, I thought it would be interesting to share this tidbit presented at the 1997 International Conference on Computer Processing of Oriental Languages in HK:

There are no common Thai surnames. Surnames were largely introduced to Thai culture only by the 1913 Surname Act. The law does not allow one to create any surname that is duplicated with any existing surnames. Under Thai law, only one family can create any given surname: any two people of the same surname must be related, and it is very rare for two people to share the same full name. In one sample of 45,665 names, 81% of family names were unique.

Europe

Denmark: Nielsen, Jensen, Hansen // patronymic ending in Norse -sen (son of)

France: Martin, Bernard, Dubois, Thomas, Robert, Richard

Germany: Müller, Schmidt, Schneider, Fischer // I wonder if umlaut's are required or they sanitize input

Ireland: Murphy, O'Kelly, O'Sullivan, Walsh, Smith // ^ same question with apostrophes

Italy: Rossi, Russo, Ferrari // From Mappa dei Cognomi website

Malta: Borg, Camilleri, Vella // 9.3%

Portugal: Silva, Santos, Ferreira // 20.65%

Russia (transliterated): Smirnov, Ivanov, Kuznetsov // 3.81%. Again, do they use the Cyrillic Alphabet or Anglicised form?

Spain: Garcia, Fernández, González, Rodríguez // 3.48% just for Garcia! Cumulative total is 9.78%. Diacritic-safe? Maybééé.

Sweden: Johansson, Nyman, Lindholm, Karlsson, Andersson // .677

United Kingdom: Smith, Jones, Taylor, Brown, Williams // 3.55%. Two sides of the same coin with regards to The States.

In Greater London, Patel shoots up to third place.

Oceania

Fiji: Kumar, Prasad, Chand, Singh, Lal, Sharma

I need to read up more on the Indian indenture system (read: slave trade) in Fiji…

South America

Argentina: Fernández, Rodríguez, González // 3.828%

Brazil: Silva, Santos, Sousa // Silva 10.5%, total at 22.2%

Chile: González, Muñoz, Rojas // 11%

Paraguay: González, Benítez, Martínez // 15.64%

8 of the top 11 surnames end with ‘ez’, the distinctive suffix of Castilian family names. The suffix means ‘son of’. This is similar to the suffix ‘son’ in English and to ‘ic’ or ‘ich’ of Slavic names .

Peru: Quispe, Flores, Sánchez // 6.35%

Yes, I know, I’ve missed a lot of countries, but 24 should be comprehensive enough. :-)

I thought a good weekend project would be to play with Robinhood’s unofficial API and zero commissions and see if it would be possible to create a profitable algo strategy. What followed was a USD$1000 deposit, a potentially restricted account, and a passive-aggressive email from Management. Here, I write about alternative brokers and how I’m planning on bypassing these restrictions. If you’re only looking for the code head straight down to the last section.

Caught with your Hand in the Cookie Jar

I know, I should have flown under the radar, been more diligent. I was! At least with the headers I sent across with each request (User-Agent/ Referrer, etc). But to be completely honest I’m sure the folks at the Robinhood headquarters have more sophisticated ways of flagging an account, whether that is through unique timing characteristics and order of operations which can be used to fingerprint the app to detect direct access. It is a very common machine learning application to detect bot automation; tech is available right off the shelf. Given their track record, I’m not surprised that they’re terrified of people using the API directly — it likely exposes them to more risk from their own corner-cutting.

I’ve read that the email I received isn’t uncommon. Adding a 5-second delay between orders seemingly isn’t enough and my account was inadvertently flagged by customer support. If you continue to use RH you need to submit requests as a regular user would, not ~390 a day (to stay under the professional trader limit).

The number is interesting because the exchange is only open for 6.5 hrs/ day (exactly 390 minutes). It seems like the limit is designed to keep trades to about one per minute.

To combat these restrictions, I export and count all of my trades. Once I have hit my monthly cap, my program doesn’t run until the following month. Anyway, I’m digressing. Now one might say, ‘why don’t you use an actual brokerage that supports algorithmic trading?’

Meaningless Metrics, Countless Choices

The problem is my strategy only applies when I’m trading options with zero commission. Some people might say if my strat breaks with this initial upfront cost, then it sucks. Whatever, it’s low risk and works for me. Let’s look at what the free market has to offer:

1. TD Ameritrade — USD $0.65/ contract

   Some back of the envelope calculations — with a lower limit of ~200 trades/ day, and a $1.30 round-trip, TD will cost me $4,000 a month.

2. Interactive Brokers won’t scam me on the spread like Robinhood, provides better support, and much better order execution/ fill. Take advantage of the tiered option pricing structure — if deposit a significant amount or have good liquidity or execute enough trades, you might be able to swing some rebates your way.

   Will unlock wider delta spreads and probably have more of an upside meaning more $$$ but IB is my main and Robinhood is my play account. Don’t mix business with pleasure.

3. But Tradier does free options for $30/ month?!

   Last I checked, they don’t actually support OTO/OCO/OTOCO orders, even though it’s mentioned in the documentation. The API also has a few rough edges, which breeds mistrust, but in my short experimenting stint, I haven’t had any notable trading failures. I did have issues with their historical options price data but didn’t bother digging deeper.

4. Alpaca, as often mentioned, is great — but no options yet.

   It handy to have an Alpaca account for their free Polygon.io API key. Both Alpaca and Tradier have a decent data API/ streamer too, although it’s not IQFeed, etc.

5. Maybe stick to Robinhood, and use an automation macro on their website. You won't have to roll the dice in regards to getting banned.

   Unfortunately, the website can get really delayed sometimes, so doing it through the API seemed more reliable. Worth looking at it if I can’t get around the aforementioned triggers the signals the tech boys in Menlo Park are looking for.

Maybe I’m looking at this the wrong way. I went back to the email and dug deeper.

I guess I’ll stick with the latter.

The Meat

A fixed interval would be easy to detect on any brokerage’s API endpoints. Whenever I’m worried about the timing of requests I end up generating a bimodal Gaussian distribution. Maybe Poisson is better, but this is simple and gets the job done.

The words and math make it sound complicated but all you really need to use is pythons random package and tune it to make it look somewhat real:

from random import gauss, random
from time import sleep
s = random()
if s < 0.3:
  t = gauss(10, 2)
else:
  t = gauss(30, 5)
t = max(min(t,120), 5)
sleep(t)

You can get crazier than this and sample from multiple distributions or look into research on user behavior to try and best mimic a user. I knew a guy who wrote his own mouse driver with laplacian curves to mimic a user and avoid detection.

I’m not that particularly invested so I’ll stick to randomness.

To offline a core:

echo 0 > /sys/devices/system/cpu/cpuN/online

To online a core:

echo 1 > /sys/devices/system/cpu/cpuN/online

Where cpuN is 0-4.

Keep in mind there's always one core you cannot disable to process interrupts.

I’ve built websites for over seven years, and over the course of that time, I’ve learned what not to say during an engagement. Below is some advice on how to handle clients and manage expectations (without overpromising and underdelivering) — via email.

This my magnum opus. My closing emails. They address all the concerns a potential client might have and answer their questions after you’ve sent over the contract.

I’d like to address a few disclaimers before we get to the body of the email:

• This email works only when you’re already in the door and the client is experiencing some form of apprehensiveness. One might expect this after you’ve sent over the RFP, or have met the prospective for a coffee chat and are moving towards drawing up the PRD. Please don’t send this before you’ve at least had a conversation.

• In this particular case, my client had sent a 14-pointer list of questions. I thought it would be best to walk her through my vision. Most of the time, shorter is better, but when you’re at the finish line I prefer to hit the nail on the head.

• Judge the importance of your email based on the value of the project: I don’t write 1100+ word emails for smaller contracts.

• You should also probably take my advice with a grain of salt. Most of my clients are word-of-mouth which gives me a great amount of leverage during negotiations.

Now the email:

Hello!

Always casual.

I understand this is your first technical project, and as I mentioned before I’d like to be completely transparent so we’re both on the same page regarding expectations and deliverables.

Assuage concerns and assure them this Agreement is mutually beneficial. This is especially important with non-technical clients — there isn’t any point in explaining the specifics — they rather prefer a simple yes or no. Technicals can be defined in the contract which for the most part will be skimmed over and is boilerplate anyway. The last sentence is the most important — I never overpromise even when you’re looking to close. There is no point in the biting hand that feeds, and the easiest way you can achieve this is by acting like something is accomplishable in the defined timeframe. Rushing to receive remittance always looks cheap.

I can be pretty flexible in regards to changes, provided they occur within the timeframe and are reasonably within the scope of the contract.

The client’s main concern was scope creep and how the project might evolve over time. I’ve worked on both sides of the tracks, and this is an inevitability in software development contracts. The Developer has to understand that some stuff sticks and some don’t, but on the other hand, the Client needs to understand limitations. I prefer to keep it flexible to appease clients, but still draw hard lines on new features.

The reason I recommended a per-project contractual agreement in this scenario was that all the materials and assets haven’t been completely fleshed out yet and I know there might be a lot of back and forth from both sides.

If you prefer to switch to an hourly rate, I can provide you a breakdown of the number of hours this project might take. To be completely honest I haven’t looked into this yet, I merely went off the deadline you provided and how long each feature might take to develop (in terms of days). I’m estimating around 200-300 hours of work. The hours required for development will not include the administrative work, sprints reviews, back and forth open communication channels that I’ve set up and most importantly the revisions and scope creep which might occur. If we go down the hourly rate I have a number of tools at my disposal that can log my hours and track productivity!

Hourly or Project-Based? The age-old question. There are positives and negatives to both sides to the argument, some developers love the idea of getting paid hourly but detest being logged and tracked. Project-based Agreements allow for some sort of financial security (based on the milestones set). I know many projects which fell through after committing 60 hours when the contract stated a minimum of 200.

To see the benefits of a client, swap the roles. Hourly rates include administrative work, weekly sprints, calls, which bump up the bottom line. If you’re sure about the project and know exactly what you want, hourly might work better. Otherwise, try and stick with the latter.

As a person engaging a client, one should be flexible and let the client choose.

From our previous conversations, to reiterate (and to answer your questions in chronological order):

1. I’ve replaced the company with your name and attached the revised contract below.

Versioning with contracts. Email attachments suffice but I have a folder of every client I’ve ever engaged with, whether completed, in progress, or failed, with files named as

[CLIENT_CODE_NAME or FILE_CONTENT]_[DATE]_[INCR_VERSION_NUMBER]

Something like this —

Contract_v1_Oct_19.pdf

I keep the editable copies for easier revisioning. Fortunately, I have a script that accepts client name, date, project cost and spits out a template filled with the relevant details so I don’t have to go around ⌘+F and replacing everything. Also, DocuSign.

2. You might re-engage me for more work, but the Support Period shall refer to any bugs or issues relating to the featured defined in the PRD and does not include any new feature development. Any updates, changes, and new features not defined in the contract will be charged on a USD$50/ hour basis. We can also look into drawing up a new contract at the end of this agreement.

My contract is ironclad. The most recent iteration is actually more in favor of Client than Developer. Again, I prefer it this way to see I’m not looking to make a quick buck but instead guide them to the promised land of sexy application development :)

But also I don’t mess around with freebies. It protects me and my contractors from being unfairly utilized even after the project is complete. The shining light of overpromising here might appeal to some, but it’s better to set expectations.

3. Answered above. Everything is included in the per-project contract and is all-inclusive up till the end of the agreement, there is no overcharge fee. If we switch to the hourly rate I’ve also provided the hourly rates above, but there still won’t be any extra fees you’ll need to worry about, only up to the hours logged and tracked.

Self-explanatory. I don’t charge extra (no overcharge fees). I actually wouldn’t even know how that would work if we have an end date for the contract.

4. It might not have been spelled out in the contract but I work keeping Agile methodology in mind. There will be continuous testing throughout the project and I’ll provide you with weekly updates along with progress over call, all while being fully transparent. If you’d like me to create weekly breakdown documentation on the progress I’d prefer to switch to an hourly contract.

Communication is key. But also display how all of this extra work (read: non-development hassle) can be detrimental to the progress of the project and will be subject to fees if you end up going the hourly route.

5. I’ll be testing on a physical device from the start — I actually prefer it compared to an emulator or virtual device. As for when you can test a working build on your own phone that should be after a month or so — you’ll be clued in on the progress.

Clients love instant gratification. They’d like to see the project as soon as possible, even if there’s barely any architecture set up. This sets them up for disappointment. Be colloquial when addressing their concern to always be clued in but also provide a date so they’re not left in the dark.

6. I created the skeleton PRD after our conversations and looking through our previous emails. I’ve made notes on features you mentioned that aren’t required at this current MVP stage:

[REDACTED] - If I remember correctly, we agreed on this being a clickable tile which opened a pop-up that intimated to users that a [REDACTED] feature would be coming soon. From our previous email correspondence on [REDACTED]:

4. Dashboard: If I recall correctly the lookbook was not part of the MVP? I recommend not having a flow to review at a later time since it will add unnecessary complications. This means user settings and displaying information to the user and making it editable which would make this a fully-fleshed out application. If this is important we can expand the scope.

I always take notes during calls no matter how insignificant. But I rarely do this when meeting in person — I prefer to be completely engaged and let the person know I’m focused. Notes help me form the basic structure of the contract and in turn a point in time which I can revisit (for negotiations).

If this is something which is essential, I can jump on a call with you today to really flesh this out completely so I know what you want. Still slightly unclear on [REDACTED].

Clearly draw a line. Redraw the contract and expand the scope if new features are added while also bumping project costs. This sort of professionalism garners respect (if at least not from a Client’s side some sort of self-respect). Phone calls are great, they provide a much-needed break from electronic mail and the feedback loop shortens.

[REDACTED] - Users will definitely be able to see [REDACTED] — otherwise, the feature wouldn’t work! I apologize — I should have been clearer. I can add [REDACTED] that provide feedback on whether the user [REDACTED] but we’re bordering on full [REDACTED] functionality now. We should try and stick to PRD.

As a Developer, one might forget that some technical specifications need to be spelled out. Apologize often, it really doesn’t take much. And if the Client is asking to flesh out a feature point to the product requirements document to cover your bases.

Regarding the PRD — once we have defined the goals and milestones — if I cross the deadline without completing the entire application to your satisfaction the additional time it’ll take to complete the project will be at no cost to you. This is usually how I work and I like to see a project through till the end.

Self-explanatory. I prefer to end contracts on a good note.

7. I have many recommendations, we can sit down/ jump on a call and discuss how it might look on your end. I’ve built [REDACTED] before so this is trivial, the [REDACTED] connection is the only cumbersome aspect.

The Client was unsure of a particular section of the application and didn’t have the mockups. I find it easier to provide examples from websites or screenshots of live applications and letting them point at features they would like to implement. At the end of the day they’re Users too, and they might know what OnboardingSection entails.

8. Answered above. Any reasonable changes can be included in the project requirements but overarching changes to the structures  If you think the project might evolve more during developed (even more than the defined PRD) we can switch to an hourly contract.

Bridging the gap once again.

9. You will have complete access, you own the entire codebase from the start! Yes, to all the above — the code will be maintained professionally and fully commented on.

Self-explanatory.

10. Any major iOS version changes during the contract period will be fully supported. I can also support 2 major versions down for compatibility — but I do recommend sticking to the latest for security.

So iOS versions rarely change over a three-month period and occur once a year. But also there is no point in me explaining this to the Client so I instead go the path less traveled and confirm her hypothesis.

11. Yes, that is correct.

A simple yes or no always works. No explanation warranted, and if needed can be addressed over a phone call instead.

12. The project end date will be 3 months from the date of signing the contract.

Clarity.

13. I understand your apprehensiveness but this is usually how I work. I’m willing to work out a payment plan for the remaining milestones but the first remittance is 50%.

I always work on a 50% upfront payment — it guarantees commitment from both sides and helps me commit completely to a project. It might be on the higher side for some developers but I’ve never once had a problem with this proposal.

14. If we want to hit the [REDACTED] deadline I recommend we wrap up the administrative work by the end of this week. A project of this complexity requires 3 months at the very minimum. Fortunately, it’ll take two weeks to set up the scaffolding and basic structure of the application and your input will be minimal. I’ll still update you on progress but I’ll really only need some assistance and hand-holding when we get to the content stage. Your UI designs provide a good starting point and we don’t have to iterate on the vision which saves a lot of time. Once I receive the first payment I’ll be locked in with you for the next three months and will be able to guide you better.

The Client wanted a few weeks to prepare the assets for the project. I explained why that wasn’t necessary and in fact detrimental to the project timeline itself.

As always, if you have any questions please feel free to shoot an email or a message over on WhatsApp — I reply much quicker!

Thank you,
Viren Mohindra

I prefer to keep open communication channels with all clients.

With elections around the corner, I thought it would be a good idea to write about a hypothetical story that occurred two years ago, and explain how easy it is to bump up voter count on online survey platforms.

Burning the midnight oil, I was on a Discord call with a few friends. Student elections were around the corner and votes were lagging for the friend who was contesting. The SurveyMonkey was linked on the channel and some non-technical friends started spamming the votes for our friend’s favor. With absolutely no verification (whether this was in the form of student email verification or CAPTCHA’s), a programmer mentioned off-hand it was trivial to write a script to turn the tables.

Selenium simulates and automates user interactions with websites. Used for unit tests, my experience with this software is to write one-off scripts to either scrape information from websites for local analysis or run regression test suites on some of my hosted applications whenever I commit or make any new changes.

In a few minutes, a script was posted in chat which leveraged Selenium to automate the voting. One interesting takeaway was that SurveyMonkey rotated options around to prevent bots from targeting specific values. One could potentially circumvent this by targeting text on the screen (instead of className, id, or name):

driver.find_element_by_xpath('//label/span[contains(text(), 'TARGET_TEXT')]/ancestor::label')

You can find the complete python code here

The script was hitting 100+ votes a minute. But the programmer realized the admin dashboard on the survey website might track and log IP addresses and they would have to rotate connections after every successful vote to hide their identity. With their VPN client, they wrote another script with a configuration file to handle this restriction, checking for server availability and performance on the fly to increase reliability and not leave their actual address exposed.

Wit this roadblock, one successful vote meant closing your previous connection, waiting for it to resolve the next IP address, and then execute, which slowed down the script to a vote every 10 seconds. Completely unreasonable cycle times. Spinning up multiple instances of the script and hosting it on multiple servers was the next best alternative option.

In a few hours, the final log showed more than 1000 votes submitted in the primary runner’s favor. Only a few hundred people were allowed to vote. The student election locked voting and launched an investigation. Nothing ever surfaced, because, in random intervals, the script also voted for other candidates. The only egregious mistake the programmer made was they ran it for too long and didn’t impose a hard limit.