As it turns out, just having data isn’t enough to get a paper out. Neither is having fun ideas about how you’d analyze it. Working on it for months on end despite multiple dead-ends, and finding a wonderful collaborator in Laura were key to finishing this paper. I hope people find the resource and paper interesting, and a jumping-off point for further work!

One of my favorite analyses in the paper is the final section, where we look at ‘deviant’ adjective-noun (AN) bigrams. Deviancy is hard to define, but it arises when a phrase is grammatically well formed but the overall meaning is weird or incongruent (colorless green, residential steak, warm equation) The term is from a 2016 paper by Eva M. Vecchi where they collect paired acceptability judgements for about 25K ANs, building distributional models that predict deviancy extremely well.

We found that Bulwer-Lytton sentences contain many more deviant ANs than control sentences (from openings to novels and humor datasets). What was surprising was that LLM generated Bulwer-Lytton sentences contain even more deviant ANs compared to BLFC entries. To derive the count of AN bigrams, I used the excellent infini-gram API to query the DCLM pre-training corpus containing over 3 trillion words. Figure 5 from our paper shows this effect beautifully: the BLFC corpus contains many more rare/deviant ANs than control sentences. Further, 80% of ANs generated by DeepSeek occurred less than 1000 times in the DCLM corpus, compared to 40% of ANs in the BLFC corpus, and 25-30% in control.

This got me thinking back to Eva M. Vecchi’s original dataset of deviant ANs. They quantified these ANs as unattested by looking at a corpus of 3 billion words— tiny by the standards of today’s pre-training corpora. What if we query their count against the DCLM pre-training corpus? The figure below validates that most of these ANs are indeed, deviant. 80% of them occur less than 1000 times in the corpus.

My counting methodology isn’t perfect. infini-gram just queries raw strings, so some entries are over/under-counted for various reasons: misspellings, polysemy (meaning its not really an adjective-noun bigram), a missing prefix/suffix, etc. But I don’t think it takes away from the story these figures tell. Then I got another idea with all the deviant ANs I had at my disposal.

So here it is — a webpage that gives you a random deviant adjective noun bigram (I filtered the list down to 3994 ANs that occured less than 10 times in the DCLM corpus). Here are some of my favorites from randomly reloading the page: bold airfield, classical bowler, informal fate, nuclear suitability.

I’ve been meaning to blog my thoughts on this memory for months, but I have a renewed interest as I make the decision to choose whom I love and marry despite my mother (and father’s) disapproval. As the emotional manipulation, blackmail and abuse reaches the worst it’s ever been, I try to remember to not treat my mother as a punching bag. Not for the reason my teacher intended (to always show grace for mothers despite abuse), but because I need to grow and flourish mentally. I understand how my mother is a product of a traumatic society and politics, but socio-political understanding can’t solve personal problems. The personal is political, but solving interpersonal conflict requires people to reflect and change. My mother can’t do that because she doesn’t want to.

A few weeks ago, Sohla El-Waylly blogged about her experiences being labelled difficult at work, and I cried reading about her upbringing:

I grew up in an extremely conservative family and only interacted with a limited group of people with similar views until I reached adulthood. My life was small and secluded. They were all immigrants from Bangladesh, away from their families and homes in a new, often hostile, country. They did what they thought was protecting us, demanding full control over their children. I never handled it well…It’s an uneasy thing believing that your mother will only love you if you do what she wants. When my parents found out I had a boyfriend in college, they stood outside my dorm room on their knees, crying and screaming, hollering my name…My mother was so embarrassed by my divorce that on my wedding day to Ham she apologized to his father for having to take me, this difficult girl, into his family.

She understands why her family behaved the way they did, doesn’t excuse it, and marches on. I don’t know how she does it, but I have to as well. Accept that my parents won’t change, will continue to manipulate and hurt me, and that I need to maintain my sanity and try to be there for them as a son. Somehow.

Here’s the FizzBuzz problem statement (from a blogpost by Imran Ghory that Tom cites in his video), that I used verbatim when prompting:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

I ran all of the prompts in the first week of December 2025, and include links to the full chat when possible.

GPT-5.1

for i in range(1, 101):
    if i % 15 == 0:
        print("FizzBuzz")
    elif i % 3 == 0:
        print("Fizz")
    elif i % 5 == 0:
        print("Buzz")
    else:
        print(i)

This is probably what I would write in response to the question. It’s a decent solution, and probably represents the average programmer’s answer to the question. I like that its clean with no comments; There’s no statement or expression so complex that needs explaining here.

gpt-5.1 chat

Gemini 3

def fizzbuzz(n_max):
    """
    Prints numbers from 1 up to n_max, with substitutions for multiples
    of 3 ("Fizz"), 5 ("Buzz"), and both ("FizzBuzz").

    Args:
        n_max (int): The maximum number in the range (inclusive).
    """
    print(f"--- FizzBuzz sequence up to {n_max} ---")
    
    # Loop from 1 up to and including n_max
    for n in range(1, n_max + 1):
        output = ""
        
        # Check for multiples of 3
        if n % 3 == 0:
            output += "Fizz"
            
        # Check for multiples of 5
        if n % 5 == 0:
            output += "Buzz"
            
        # If output is still empty, it means n is not a multiple of 3 or 5,
        # so we print the number itself.
        if not output:
            print(n)
        else:
            # If output has "Fizz", "Buzz", or "FizzBuzz", print the string.
            print(output)

# Run the function for the required range (1 to 100)
fizzbuzz(100)

I didn’t ask for a function, but Gemini wrote one anyway. Google seems to have focused on training Gemini models (since at least 2.5) to always output code that’s extremely well-commented. I’m surprised there aren’t type hints¹ and emoji sprinkled all over the comments and print statements, but I guess it reserves that behavior for more complex coding challenges. Both of these are almost certainly behaviors the model ‘learns’ during human-preference alignment: annotators/users just prefer responses with emojis and lots of comments. I don’t.

It did use string concatenation to solve FizzBuzz though, which is a smarter way to solve the problem over if-elif-elif-else.

gemini chat

Claude 4.5

Claude Haiku used string concatenation and was thankfully less verbose like GPT-5.1:

for i in range(1, 101):
    output = ""

    if i % 3 == 0:
        output += "Fizz"
    if i % 5 == 0:
        output += "Buzz"
    
    if output == "":
        output = str(i)
    
    print(output)

haiku chat

Claude Sonnet, despite being the larger, smarter model, chose to solve the problem using if-elif-elif-else:

for i in range(1, 101):
    if i % 3 == 0 and i % 5 == 0:
        print("FizzBuzz")
    elif i % 3 == 0:
        print("Fizz")
    ...

The responses from GPT-5.1 and Claude were short and plain, and I prefer them both to Gemini. However, both these models do write extensive comments (and emoji) when they are tasked with bigger coding challenges, which is smart product design on OpenAI and Anthropic’s part.

sonnet chat

DeepSeek-V3.2

DeepSeek-V3.2 gave me 3 different solutions. A solution using if-elif-elif-else, another using string concatenation, and this one-liner:

for i in range(1, 101): print("Fizz"*(i%3==0) + "Buzz"*(i%5==0) or i)

I hate one-liners and code that’s trying to be too cute or smart, but I’ll excuse this. Barely. Multiplying a boolean with a string and then taking inclusive or with an integer… I think this is Python’s fault for allowing such a monstrosity.

deepseek chat

What did I learn?

A student in my introductory Python class asked if there were any accuracy differences for code generated by different AIs, and I said no, but there were differences in style. After this little exercise and more experience with coding agents, I’m not sure style is the right word. The code samples above are stylistically different, but the differences come from design decisions by the companies to boost engagement.

Gemini adds extensive comments because Google thinks you’ll like that code more and it will get you to use Gemini more. DeepSeek generates 3 solutions probably for the same reason. Claude and GPT-5.1 are more adaptive in their responses, but they exhibit similar engagement boosting tactics². Ultimately, I think this exercise gives a peek into what the latest LLMs from these companies actually are: products designed with the purpose of increasing ‘engagement’. They just want us to keep using them.

Update: I took it one step further and made a benchmark out of LLMs playing FizzBuzz. Some mildly interesting results!

The paper is a short read, I hope some people find value in it. But I wanted to use the paper as an excuse to revisit my very first post on this blog. It’s been five years, has the phenomenon of nuance rising slowed down in ACL publications?

Nope. At the current rate of doubling, over half the papers published in *CL venues will mention nuance/fine-grain by 2040! I can’t wait to write the follow-up to this post in 2030 to see if we’ve reached peak nuance.

Again, I don’t hold a value judgement one way or another and at least 2 of my articles contribute to this phenomenon. I think words like ‘nuance’ and ‘fine-grain’ are just words we attribute high value to as researchers and use even when we don’t need to or have to. Words also fall into and out of style — but I do think a lot of us also fall into the nuance trap when we study research questions to make our papers sound more appealing.

One undeniable positive to come from all the hype around ‘AI’ is much better developer tooling and frameworks for embeddings and neural networks now than in 2017. With Qwen3 multilingual embeddings and Gemini CLI¹, I could quickly prototype the first idea that popped in my head: Build a web app that uses a multilingual embedding model to find relevant Thirukural couplets (in Tamil) for user queries in any language. That’s what ‘Kural for your question’ is. I’m pretty happy with the end product, but the retrieval of relevant kural couplets itself with cosine similarity of embeddings is pretty underwhelming.

Visualizing the embeddings of all 1330 kurals in 2D using UMAP gives an idea why. There aren’t really any meaningful clusters, with only the Book of Love showing some clear separation from the other two books. The similarity search works sometimes because of certain key words, but lacks any understanding of the intent of the question. For the question What are the duties of a son to his parents? (an undying question for me), only one of the 3 kurals deemed relevant is about the parent-child relationship at all. The larger Qwen3 embedding models might work better, but model training frameworks and data mixtures are more biased towards real-world use cases — my niche, little idea probably doesn’t mesh well with what Qwen3 Embedding was trained to do.

Still feels good to build something, even if underwhelming 🙂.

A few lines of dialogue caught my attention towards the end of the movie. The main character is solving a crossword puzzle when she utters the meta-linguistic clues out loud:

“Danger.” Six letters.
Threat.
“O positive.” Ten letters.
Blood group.

The above lines are from the English subtitles that I was reading, but Fallen Leaves is a Finnish movie. I wondered how the subtitle translators shifted between cultures when writing the English subtitles for these language-specific clues, so I naturally looked up the original Finnish subtitles:

“Vaara.” Kuusi kirjainta.
Uhka.
“O Positiivinen.” Kymmenen kirjainta.
Veriryhmä.

Kuusi and Kymmenen are the cardinal numbers six and ten in Finnish. But the Finnish word for bloodgroup, Veriryhmä according to the subtitles (and Google Translate) is 9 characters long. However, Finnish plurals are frequently formed with the -t suffix, and the captions might have missed the plural, or the character might have simply said the singular. But, what about Uhka which is just 4 characters?

It is at this point while writing this blog post that I realized she’s solving an English crossword puzzle, and just reading the clues out loud in Finnish🤦🏾‍♂️. So much for getting excited about an interesting translation puzzle in the wild.

Sky-high overview

The first misconception I had to forego was the link between concurrency and parallelism. Rob Pike describes the distinction in a succinct way:

Concurrency is about dealing with multiple things at once. Parallelism is about doing multiple things at once.

The dealing with part is important! I didn’t understand the need for this because of how removed I’ve been from low-level programming. YouTuber Core Dumped has [a really good video about concurrency][coredumped], and his explanation finally helped me understand why concurrency is not only important, but crucially the default. Processors execute one instruction at a time — but modern operating systems run hundreds of active processes at any given time. The Operating System (OS) is in charge of figuring out which process should get access to system resources. Modern CPUs are so fast that we’re under the illusion that all processes are running simultaneously²

This fundamental truth about deadling with multiple things that need to get done applies to individual applications/programs as well. Thankfully, application developers can rely on the APIs and abstractions provided by the OS to handle concurrency. For developing applications on Apple platforms in 2025, this means adopting language support for concurrency in Swift.

For my simple app DeTeXt, there were two instances where I needed to deal with concurrency in my code:

• running image recognition with my CoreML model as a separate task on the main thread.
• showing a temporary toast-style pop-up when the user copies a symbol/command/unicode code-point. Once again, this is a task that runs on the main thread.

But what’s a thread? Or a task? And why am I running things on the main thread?

Threads

A process is a program in execution. It has its own program counter, register information, and memory space. However, programs themselves need to do multiple things at once within them. Threads are the abstraction that almost all OSes have settled on to handle concurrency within a process. Every process has a main thread, which is the initial thread where all work first happens. The OS typically initiates subsequent threads based on developer instructions.

For a user-facing mobile application the cardinal rule is that all user interface (UI) work must happen on the main thread. Updating the user interface is a short-term operation that immediately affects the user experience; Users can tolerate waiting for a large download or file save, but the app itself should never crash or lag. Critical operations happen on the main thread, which has OS-level priority for the application.

DeTeXt is a pretty simple app — it doesn’t download or upload anything, all symbols and images are loaded on start-up, and they’re only 10MB anyway. Implementing concurrency in DeTeXt simply meant identifying (and marking) asynchronous functions and possible suspension points (where possible long-running tasks can occur), and instructing the OS to run all asynchronous functions on the main thread, which it was doing anyway.

Actors and Tasks

Swift’s concurrency model doesn’t let us work with threads directly. Instead we work with 2 abstractions — tasks and actors.

A unit of work that we need to handle with concurrency is a task. Fetching web resources, reading/writing from the file system are traditional examples of a task suited for concurrent thinking/processing. For DeTeXt, I defined 2 tasks that encapsulated the 2 asynchronous functions³ mentioned earlier:

• Taking the drawing from the on-screen canvas, pre-processing it, and processing it through the neural net that calculates probabilities for every symbol. The probabilities are stored in a reference type marked Observable —any changes to the underlying data send notifications to SwiftUI views that observe it.
• Displaying the name of the command, or the symbol itself and showing it as a toast-style pop-up on screen for a set, constant period of time, then automatically dismissing it.

Now both of these need to run on the main thread, since they update the UI in both cases. However, both tasks can take undetermined amounts of time to finish. The CPU/GPU/Neural engine might be clogged up doing some other intensive process (very unlikely but possible), and the toast task needs to pause for a set amount of time before finishing. Implementing this in DeTeXt couldn’t have been simpler: package the asynchronous function call to the CoreML model and the toast suspension/sleep as tasks. Now, we need to ensure that both tasks run on the main thread, for which we turn to another abstraction: Actors.

Actors are objects that ensure that only one function has access to mutable data at a time. Their role in concurrency is to avoid data races. The Main Actor is in charge of updating all the data that updates the UI. Since the only mutable data in my app pertains to the UI, I simply needed to mark the two asynchronous functions with the @MainActor attribute, instructing the OS that while there may be suspension points in these two functions, both of them impact the UI, so ensure that these functions only run on the Main Actor. The Main Actor abstracts over the main thread — they’re very similar, and the differences have more to do with low-level implementation details.

Fin

I learned a lot about concurrency and asynchronous functions while re-writing my app to use Swift’s new concurrency features. You can view the actual code changes on the GitHub repo of course. To be honest, writing this blog post took more time than actually learning and implementing Swift concurrency in my app! I love explainer blog posts and videos, so I figured I’d give a shot at writing my own, for me.

Dutifully, me and a few others raised our hands to this pointed question by our social sciences teacher. I believe we were in ninth or tenth standard, so in our mid-teens. Her reply to the handful of raised hands was just as biting — I don’t believe you.

She was right; at least, about us not being truly proud. We raised our hands because we knew it was the right thing to do, not because we were truly proud of our country. I couldn’t have described with any words what ‘being proud of your country’ meant at that time. There were empty quotes and calls to duty, but duty to whom? My recollection was that we were taught to serve our country by not questioning authority or tradition. Pride in one’s country, as my social science teacher (and most adults around me at the time) understood it, precludes questions, criticisms and introspection of cultural practices. But pride that requires conformance, fear or unquestioning respect of authority is not worth having. Considering the strict, authoritarian and traditional environment of school where our teacher was an active participant, I’ve no doubt this was the pride she sought. I know now that I want no part of this pride.

She wouldn’t believe me today if I said I was proud of my country, because we diverge on what pride means. I take pride in the people in India and around the world who question authority, injustice and fight for freedom from suffocating power structures. I’m proud of people who try to interrogate their privilege and try to unlearn their indifference to suffering. One day, I hope to be truly proud of my country and myself, as I do the necessary, hard work that is required of anyone walking down this path.

Viewing film art as perpetually subject to update and correction, political or other, says something about the director’s relationship to his work…Director’s cuts are both futile and wrong-headed. In a way, they redress artistic grievances by treating movies as mere product. In this country, an inalienable right to change as an artist doesn’t extend to changing art and, therefore, art history.

I don’t think the argument is that Director’s cuts should be outlawed as a concept — rather, treating cinema as subject to revision and change (which can be marketed) has consequences for the creative pursuit of film-making and how the audience responds to the art-form. Which brings me to this reply by John Gruber to Ken Kocienda’s post on Mastodon:

For as long as I’ve known him — and I’ve known him going on, jeez, almost 25 years now — @brentsimmons has said his favorite part of programming is deleting code. I think that’s perhaps the fundamental thing about programming that makes it unique as an art form. No other creative medium I can think of has that quality.

While this reply is about deleting code, Kocienda’s original post describes the way most (good) software developers view their code. We learn not to grow attached to our code, and to focus on constant updates and refinement. But what does this say about our relationship to our work, as Solman raised about Director’s cuts?

I’m not going to argue that programming isn’t art; I’ve already written about how the Is this art? argument isn’t very interesting. However, two things are true at once:

1. Programming is fundamentally revisionist — you have to maintain and update your code-base or it usually won’t work sooner or later. There’s always bugs and inefficiencies to fix. A software project not being actively maintained means death. Programming might have artistic qualities and it is definitely a creative pursuit — but the intended product of your program has to work and be stable for some time to be useful.
2. However, the fact that programming is fundamentally a revisionist creative pursuit means there are consequences to embracing it whole-heartedly. Big Tech releases half-baked software all the time under the Beta label. Video-game companies release unfinished video games and market it as Early access. Just because software is amenable to updates and refinement doesn’t mean our standards for good software releases should go down, but there’s where we are now.

Drawing a connection between the unintended consequences of Director’s cuts and revising code gives us a new lens to understand how we approach building software, but it shouldn’t be taken as a directive to update or revise code less. I think knowledge of the unintended consequences of the nature of one’s creative pursuit or medium is valuable in of itself. It helps us make better decisions when writing code and building software.

Most systems are designed with the idea that they are supporting users who prefer to put in as little effort as possible when accessing information. This is certainly a desired characteristic of an IA system. However, we argue that it is not always advisable to minimize this effort, especially if it takes away the ability for a user to learn, for instance due to lack of transparency. We believe, and as others have supported, that certain amount of ‘friction’ in an IA process is a good thing to have.

The paper’s focus is on search and information access, but one could extrapolate it to any user-facing technology, and I’ve been especially concerned by its implications towards software and user-interface design. The paper they cite by Jeremy Pickens is a good read as well. However, I don’t think either paper truly captures the scale of the problem, which precedes AI.

Friction is impressed upon developers and software designers as something to avoid and design away first. At least, that’s how I understood it (even though I hadn’t mapped the lexical item friction to this concept) when I learned programming and app development by myself. Formally trained UI/UX designers, or those who’ve worked in the area for years clearly realize its utility and value. However, I’d contend that eliminating friction, not making users read/think/wait, and designing easy-to-use interfaces that delight users is the overwhelming default mode of dealing with friction in software design.

The Verge’s interview of Google CEO Sundar Pichai back in May 2024 exemplifies this thinking. Pichai acknowledges the trade-offs inherent in providing users with instant LLM-generated answers at several points — but that hasn’t stopped them from stuffing AI answers and summaries everywhere. Choosing not to include AI summaries, or even introducing the least friction into the process, would promote a healthy web ecosystem and have other social and individual benefits in the long term (as Shah & Bender argue in their paper). But someone else would just do it and threaten Google’s dominant position in tech. The Big in Big Tech isn’t just a modifier — it’s fundamental to what these companies are and how they operate.

Perhaps the default way of conceptualizing friction in software design served users and society well until the early 2010s. But the smartphone, ubiquitous internet, social media, and now AI have changed how I think about it. Sometimes you have to frustrate individual users for the well being of the community. Unfortunately, the default path is heavily incentivized, since there’s a lot of potential money (and power) at the end.

It’s been a while since I’ve written any blog posts. When I think about my heroes, a few names come to mind: Christopher Hitchens, Jessica Valenti, Chomsky, Ambedkar, and Tom Scott. I see the failings of some of them now, but I still consider them my heroes because I was obsessed with anything and everything they wrote or said that I could find. When I realized I look up to Tom Scott, I understood why I was obsessed with all of them — they kept writing and working on new projects. They were never satisfied. I’ve been paralyzed internally trying to find satisfaction, or re-gain my attention, when the answer is quite simple; There is no closure, I just need to keep working on new projects (and writing).

Publishing DeTeXt on the App Store was exhilarating, because I made it. The last few years of my life have been the best, and the worst. But the numbness that I recognize internally, can be traced back to the root cause of failing to make things. Anything. This blog post, an app, research, kozhukattai…I should be able to say ‘I made this’ much more often.

What the fuck does authentic mean anyway? … The term ‘‘authentic’’ … is essentially meaningless. ‘‘Authentic’’ when? ‘‘Authentic’’ to whom? But it sounds good and wise doesn’t it?

Anthony Bourdain. ‘‘14. Alan Richman is a Douchebag’’. Medium Raw

Some things leave a clear semantic trace in my mind — I don’t remember the exact words¹, nor the circumstances, but they leave a dent. In my mind, Bourdain’s acerbic takedown of a journalist questioning the authenticity of Creole cuisine is up there with Siracusa expounding on sports, or the final shot in Ozu’s The Only Son. They contain multitudes in so little, and not just because they’re akin to pithy aphorisms — they nudge me to research, read, learn, and ponder on their meaning endlessly. They enrich my life, my work, and relationships, even if it involves uncomfortable introspection.

Questioning the authenticity of food especially evoke a deep-seated rejection fermented by years of pondering on Bourdain’s words, and my own cowering behind that empty word. This isn’t to say one can’t criticize food (or any cultural product), but if you see fit to judge and expound on it, be decent enough to admit what you’re doing and why — maybe you wanna feel superior, or occupy space in the conversation, or be seen. You can do that without empty gestures towards authenticity.

Funnily enough, what I yearn is for people to be genuinely authentic (in the sense of be honest about what you’re feeling or wanting) with me.

It is easy for an American intellectual to deliver homilies on the virtues of freedom and liberty, but if he is really concerned about, say, Chinese totalitarianism or the burdens imposed on the Chinese peasantry in forced industrialization, then he should face a task that is infinitely more important and challenging—the task of creating, in the United States, the intellectual and moral climate, as well as the social and economic conditions, that would permit this country to participate in modernization and development in a way commensurate with its material wealth and technical capacity.

— The Responsibility of Intellectuals, Noam Chomsky

I have lived in the US for over 6 years. I feel like I’ve come of age here — certainly I didn’t dwell on my responsibilities much when I lived in India. I think I want to live here, even though I don’t belong (and never will), but what of this responsibility I feel (with my freedoms) to do something about injustices in India?

Answering these questions¹ are important to me, since it clarifies the root of disagreements with family, friends and colleagues, and how to effectively fight injustice in India. I don’t have a great answer yet — but the strong whiff of bullshit from me when opposing injustices in the US when India made me is a good enough motivator to find an answer.

So what are the responsibilities of the non-resident intellectual beyond seeking the truth back home, and the historical context of the truth? Supporting intellectuals and activists back home must be one of them, if not the foremost one.

What is to be done? The fix-the-system crowd wax philosophy about how loverly it would be in their world where we’re one with nature, and how we need to organize and fight corporations. My patience is thin with these pleas, especially with this issue. What’s more likely to happen — that I discipline how I allocate my attention, or that the internet economy re-aligns itself through self or external regulation around the amorphous concern of ‘attention’?

I’ll take my chances with disciplining myself.

I can’t find it right now, but I remember an old piece of advice — possibly in the HIG — that said that items mathematically centred vertically tend to look like they’re lower than they are. The suggestion was that a visually vertically centred item typically needed about twice as much space below the item compared to the space above it.

Page 184 of Apple’s HIG guidelines from the 2003 edition:

[the] distance from the bottom of the window to the top of the Dock (if it’s at the bottom of the screen) should be approximately twice the distance as that from the bottom of the menu bar to the top of the window.

Matthew Butterick on centering page contents using margins:

To make the text block ap­pear cen­tered ver­ti­cally, try mak­ing the bot­tom mar­gin about a 0.25″ larger than the top mar­gin. Oth­er­wise, the text block can look like it’s sag­ging.

Finally, David Lynch as John Ford to Sammy Fabelman (Spielberg):

Where is the horizon?…Where is the goddamn horizon!?…When the horizon’s at the bottom it’s interesting. When the horizon’s at the top, it’s interesting. When the horizon’s in the middle, it’s boring as shit.

Only one Thirukural of the hundreds we had to memorize in school has stuck with me. I don’t remember the words of course, only the rough meaning:

A son’s duty (to his parents) is to ensure that others speak highly of/praise him to his parents.

As I write this, I probably read this kural 12 – 15 years ago. As it turns out, this wasn’t one kural leaving a trace in my memory, but two. Both are from the Book of Virtue, in the chapter dealing with children. The first, kural 69 is about one’s duty to their mother¹:

Eendra Pozhudhin Peridhuvakkum Thanmakanaich
Chaandron Enakketta Thaai

A woman rejoices at the birth of a son, But even more when he is praised.

Kural 70 is about duty to the father:

Makandhandhaikku Aatrum Udhavi IvandhandhaiEnnotraan
Kol Enum Sol

The son’s duty to his father is to make world ask, ‘By what austerities did he merit such a son!

I’m glad to realize now, in my wise age of 29, that even Valluvar had to resort to padding. Usually I lament my memory’s fallibility, but in this case, I’m glad my mind saw the redundancy and reduced them to one thought.

Unfortunately, the amount of mindspace this thought occupies is more than two or four verses, and will only grow with time. Every day is a slow realization how true the verses are when they shouldn’t be, and a reminder how I’ve not done enough, even though I’m enough.

(1) I show Mr. X a photograph of Mr. Y displaying undue familiarity to Mrs. X.
(2) I draw a picture of Mr. Y behaving in this manner and show it to Mr. X.
I find that I want to deny that in ( I ) the photograph (or my showing it to Mr. X) meant,, anything at all; while I want to assert that in (2) the picture (or my drawing and showing it) meant something

The example, and his explanation that follows, leads him to come up with a succinct definition of non-natural meaning:

A uttered x with the intention of inducing a belief in B by means of the recognition of this intention.

Breaking down the different intentions in the definition is helpful. There are 3 distinct and inter-dependent intentions:

1. A intends to induce a belief in B.
2. A intends for B to recognize their intention to induce said belief.
3. A intends for B to gain this belief by means of recognition of the first intention.

I’ve been thinking a lot of this definition of meaning in the context of ‘AI generated’ art. The usual suspects are confused why ‘AI generated’ art hasn’t taken off. The obvious answer is that it’s because we know it’s not made by a person — if we know something was solely software generated, we lose our ability to connect. But is this the same thing as saying we know that the generated art is meaningless? A work of art is not an utterance, so Grice’s definition will need to be adapted, but what needs to change?

Originally, I was going to go conclude this post nicely by going through the intentions when a person makes art. But I realize now that this would be unsatisfactory without answering the questions I just raised, and answering those would be a much longer post. I’ll come back to this one day, but revisiting the paper, and writing down the definition was fun.

This is an evolution even from my most recent post, where I was still quoting the Litany against fear for my research. Of all things. At this moment.

So what’s new since then? Two papers, a talk at EACL that was received well by a few choice people, and most importantly, an idea for a long-term research niche. Generalization, in Language, and across languages and communication mediums. A rich vein I can harvest for years to come. If I stay in academia.

Everything around you that you call life was made up by people that were no smarter than you.

I’m thinking about this today, but in a different context than what Hitchens or Jobs ever intended (probably). All the bigotries and unjust hierarchies that surround us are manmade. The ‘traditions’ that we impose on others, even those we love are manmade. Culture is manmade — just because something is part of ‘our culture’ doesn’t make it moral. I wish it was easy for everyone to accept this truism. Until then, I’m resigned to suffering on account of the cultural obstinacy of loved ones, although I console myself with the thought (or promise to myself) that I would never impose the traditions and customs that comfort me on my children. The cycle ends with me.

Unrelated to all this, I feel…weird quoting Hitchens so often when I write here. One day I’ll write about my heroes, Hitchens (one of them), and how my thoughts and feelings with regards to him have evolved over the years.

The most troublesome arguments to try to make are those that are both very specific and limited (and are by nature of that more or less non-controversial) …

I’ll need to re-read his post multiple times to understand his actual argument — I find that often true of him, and the rationalist community (even though he hates them). Until then, I want to address just these lines, and the sentiment behind them, which frustrates me in many aspects of my life.

Yes, specific and limited arguments are less controversial, but they’re not less interesting. I firmly believe being specific and narrow in your line of questioning or argument (or even interests) is a good thing. It usually doesn’t carry the baggage that RC is afraid of — or if it does, just state that you don’t want to generalize outright and move on.

I see this with research all the time; work on a niche subject or topic needs to be generalizable beyond the domain of analysis, or the authors will have a hard time with reviewers. You can’t just study, for instance, how Eagles fans talk about Cowboys fans online. What about other teams? What if its not online? What if its not sports? I get that a research question is always grounded in past and contemporary work, and we need to be cognizant and report that, but I think we’re overdoing it in Computational Linguistics and NLP right now. Anything and everything you do has to be generalizable, or fit into the existing work perfectly.

Not that this is a unique problem with research in my area, or research in general. Arguments online and in-person get derailed because we forget the specific circumstance or incident — it’s so much easier to generalize without actively wondering if it makes sense to generalize. I do this too, and am trying hard everyday to counter this (I suspect) deep-rooted tendency in my mind to find patterns, over-generalize, or at worst, stereotype.

I’m not happy with this post as it stands; I feel like I’m jumping between the specificity of arguments, questions, beliefs, and thoughts. That’s fine — I’ll always be thinking about this, and writing my thoughts down clarifies them for me. Maybe in a few months, I’ll revisit this topic. But today, I do want to mention and link to two comments that resonated with me, and possibly catalyzed my thinking on this topic. First, from Christopher Hitchens in this video (starting from 18 seconds in), on how to ask a question:

Here’s a piece of advice about asking a question. Try and narrow it down a bit… if you give me too much to chose from, you’re likely to get everything or nothing, and I certainly can’t give you everything…

Also, here’s Nikki Giovanni, starting at 21:10, on being parochial in her activism (the entire interview is amazing):

… because I tend to be parochial, for one thing, and I tend to care about Afro-Americans … It’s very parochial because I don’t care about my third world brothers and sisters … If I deal with my block, and you do with your block, we’ll have two good blocks.

Being parochial in one’s passions might seem like a different thing from what I was talking about earlier, but they’re not separate in my mind. I don’t think I’m overgeneralizing here to lump these two together. I think what they’re both saying succinctly describes the value of being narrow, be it in questions, arguments or interests.

I write in this space in order to put ideas out there quickly, often in an unpolished form. I like the idea of sometimes making my mistakes in a relatively open forum. Unlike an article, or a book, which are years and years in the making, this substack newsletter capitalizes on my strong desire to write, sometimes quickly: to seize a few hours or even minutes here and there, and try to articulate ideas which have sometimes nagged at me for a decade or more without ever reaching fruition. By and large, I think by writing.

All my agonizing over what this blog is for, and what I should write about, and why, and someone else puts it down in better words than I ever could. Like it does for her, writing clarifies and helps me think. I don’t however, think by writing right now — I would, however, like to. I aspire for it, for its own sake.

It was only recently that I realized that sometimes, frayed and broken relationships cannot be fixed. I could apologize, make up for my mistakes and shortcomings, undo all the harm I’ve caused, and yet. I think I learned that a sincere apology transfers the responsibility onto the other person to forgive, forget, and accept you back. The truth is that there is no moral or natural law that mandates people forgive, forget and accept after a sincere apology, nor would I want there to be.

All of this may seem obvious in hindsight, but this is my blog after all. I can write about the trivial as if they were life-altering revelations.

It struck me that today’s AI assistants (Alexa, Google Assistant, Siri) are all based around having conversations. If these systems ever approached anything close to human intelligence and common-sense, perhaps having a conversation is the best way to interact with AI…maybe the best way to interact with artificial intelligence is the same way we interact with other people — using conversations.

Austin Henley, in a lovely blog post from a few days ago:

A great user interface guides me and offers nudges.

Couldn’t a natural language interface help with that?

Certainly.

But not as the only option. Probably not even the main interface.

My first thought after reading this was Why didn’t I think of that?! Everything Austin says about software and user interface design I (sort of) knew 2 years ago. People know what they want — but they can rarely express it well, especially using words. Relying solely on natural language to gauge user intent is lazy, and will lead to poorly designed software.

A brief digression to put down my perspective on software and UI design: I believe that self-imposed constraints and strong opinions (weakly held) are good and necessary towards building well-designed software. This path will lead to your product not appealing to everyone, but that’s okay! My favorite apps are opinionated and have relatively a small/medium-sized customer base; but that’s all they ever wanted. It always frustrates me when a new service/product clearly has aims to be ‘the next big thing’ from the start. Why do you need millions of users/customers? If you’re one person building something new, you only need thousands.

Circling back: my original post was about the best way to interact with virtual assistants like Siri. Natural language is the only interface for this class of software. They’ve been around for a decade, and while useful (especially for accessibility), they haven’t revolutionized how we do things everyday, nor have they opened up new opportunities, I think. How do we make people successful at using computers using a lazy interface?

I hope I’ll play a role in answering that question one day.

I feel relieved and optimistic at this juncture in my Ph.D. And yet, the fear is still there, receded in the background. I can envision scenarios where it engulfs me again, but not many. In any case, I will face it when it comes, and only I will remain after it passes.

Will my research program lead anywhere, change anyone’s mind, or even be one that anyone apart from me finds interesting? Probably not, but I’ve come to realize that all that matters is I do the work, do it well, and that I grow during the process. Like programming languages, I’ve come to learn that the research programs that gain traction in a community, especially a relatively young one like Computational Linguistics, have little to do with the program’s promised benefits — external factors like the progenitor’s community and its ease of use play a big role. I can only aim to do the best on what I control — ensuring that I pursue my research questions honestly and rigorously.

Addendum

I was a little too optimistic with giving my own name to my focus of study, but it was good in the end to get pushback on Generalized Intergroup Bias or GIB. I study intergroup bias; how is in-group language different from out-group language? That’s the one sentence summary of my dissertation, and I’m quite happy with it. My own term and acronym would have just made it too confusing.

I’ve circled the disclosure control next to the ‘Shared With You’ section that lets you expand and show more items. Now consider the same New Tab page on iOS Safari:

On the Mac, the control for expanding ‘Shared With You’ is a downwards-pointing chevron; it becomes upwards-pointing when you expand it. On iOS, it is downwards-pointing after you expand it, while it points to the side when the section is collapsed.

The first thought that came to mind when I noticed the different direction of the disclosure links was Gruber’s post on what disclosure controls ought to do:

In the iOS/Mac style, a right-pointing chevron (or triangle, depending on the OS) indicates the collapsed state, and a down-pointing chevron indicates the expanded state.

Based on that article, I thought I had found another instance of Apple’s own apps not following the HIG, but that isn’t the case. Based on my reading of Apple’s HIG, this control is a disclosure button on the Mac, but a disclosure link on iOS. Now I have two questions:

• Why do disclosure links show state and imply action while disclosure buttons show action and imply state?
• Why is it a button on the Mac and a link on iOS?

I feel like there ought to be a clear answer to the first one, but the examples in the HIG don’t elucidate the differences to me. Regarding the second question, I think its a mistake by whoever designed it. Disclosure links are all over macOS and they do exactly what this disclosure button in Safari does: expand a list of items to show more. Why should this control be different?

In the ordinary moral universe, the good will do the best they can, the worst will do the worst they can, but if you want to make good people do wicked things, you’ll need religion.

I used to believe this to be true, as I did many things that Hitch wrote. The statement is obviously false, as well stubbornly simplistic. I’m not too concerned with the semantics of the quote, or the soundness of its argument, but I do think about it a lot, especially at this moment in my life.

I have no answers. I only have compassion for those who fear the intolerance of society, and loathing for the intolerant.

Me: Does causation imply correlation?
ChatGPT: Yes, causation implies correlation. If one event causes another, then there must be a relationship or correlation between the two events. Causation is the relationship between two events in which one event causes the other to occur. This means that if event A causes event B, there must be a correlation between the two events, because the presence of A is necessary for B to occur.

Correlation doesn’t imply causation is a dictum that’s been repeated to death in statistics classes everywhere. But does that mean causation necessarily implies correlation? Initially, I too would have had a similar answer to ChatGPT above, but as I’ve discovered, the answer depends on what you mean by ‘correlation’. To be precise, and its always good to be precise with math and statistics, causation does not necessarily imply correlation.

Correlation generally refers to linear correlation, as in Pearson’s correlation, which only measures linear relationships between variables. However, its perfectly possible to have one variable to have only a non-linear dependency on another, but the corresponding linear correlation would be zero. While we do have non-linear correlation measures, mutual information is the measure that we want in our new dictum: Causation implies high mutual information.

I’m comforted that I’m not the only one who made the error of not being clear and precise with my words (and thus my understanding) in statistics. Nobel prize winners do it too.

An aside about ChatGPT

The first time I tried the causation prompt with ChatGPT, it didn’t understand the question, and proceeded to give a rote explanation of why correlation doesn’t imply causation. The fact that these models are easily tipped off by this sort of diversionary tactic (as Venelin explored in the adversarial QA paper) dissuades me the most from the idea that there is latent language understanding and meaning that can arise from the simple task of language modeling. However, when I tried it again before writing this post, ChatGPT got it right! I managed to come up with another one that tricked it along the same lines though:

Me: What came second, the chicken or the egg
ChatGPT: The question of which came first, the chicken or the egg, is a classic conundrum. It is often used as a metaphor for a problem that has no clear solution, or for a situation in which it is difficult to determine the cause and effect. In this case, there is no definitive answer, as the question itself is based on a false premise. Chickens hatch from eggs, so a chicken must have come from an egg. However, the egg that the chicken came from must have been laid by a chicken, so the chicken must have come first. This cycle continues indefinitely, making it impossible to determine which came first.

I am a believer at a high level of the 10,000 hours concept…You literally have to put in 10,000 hours of work…

Now, while Malcolm Gladwell does peddle a lot of crap (see Gelman), the 10,000 hours idea is reasonable. Keep working on something for a few hours everyday for a few years, and you’ll probably be really, really good at it at the end. I think it’s good advice, to which I’d add a small aphorism. The best way to learn a technical concept in software, machine learning and related fields, is to build something with it. I learnt SwiftUI and Combine by building DeTeXt. I learnt Javascript by building websites and web apps. Most of my understanding of transformers and modern deep neural nets comes from trying to use them in my research projects. So yes, spend a lot of time consistently on a topic, and try building something while you’re learning it.

I want to be a good writer, and the only way forward towards that goal is to write more. But what am I building? This blog? This doesn’t feel like a cohesive unit that I’m building, not that it has to be one. Perhaps I’ll keep chipping away at the 10,000 hours by writing regularly here, until I find something I want to build.

Initially, I had trained the MobileNetV2 neural network for DeTeXt’s symbol recognition engine on a 4 GPU cluster. Today, I did the same on my M1 Pro Macbook Pro running on battery, thanks to tensorflow-metal and the 16 core GPU. I wish I had documented the training better on the cluster, but I do have the training time for the M1 Pro: 6 minutes per epoch, for training a simple image classification neural net on over 200,000 images. In 1 epoch, the network reached an accuracy of 65%. Six minutes to train a complex neural network with over 7 million parameters, on battery power. Wow.

• No hacker news
• No reddit
• No twitter
• No MacRumors, ArsTechnica, Techmeme, AnandTech…

These activities are fine (but with some conditions):

• Podcasts (only while running, or at the gym)
• YouTube (during meals)
• NetNewsWire (once a day)

Why am I doing this? For the same reason CGP Grey spelt out in his blog — I want to spend more time doing things I care about: research & study, reading, writing, cooking, building easy-to-use and delightful software tools and apps, and watching films.

Update

This is much harder than I thought it would be. I find myself replacing the time I would sink into the outlawed activities with others that seem ‘alright’ — Wikipedia, Letterboxd… There isn’t any magical solution, other than to be mindful with what I’m doing.

Many years ago I saw an ad for a running shoe (maybe it was Reebok?) that said something like “At the New York Marathon, three of the five fastest runners were wearing our shoes.” I’m sure I’m not the first or last person to have realized that there’s more information there than it seems at first. For one thing, you can be sure that one of those three runners finished fifth: otherwise the ad would have said “three of the four fastest.” Also, it seems almost certain that the two fastest runners were not wearing the shoes, and indeed it probably wasn’t 1-3 or 2-3 either: “The two fastest” and “two of the three fastest” both seem better than “three of the top five.” The principle here is that if you’re trying to make the result sound as impressive as possible, an unintended consequence is that you’re revealing the upper limit.

My first thought was naturally that this sentence is an instance of a scalar implicature. The unintended consequence is a scalar implicature that arises because of a violation of the maxim of quantity. Reebok doesn’t want to say “The third, fourth and fifth fastest runners of the marathon were wearing our shoes” (in the worst case), so they choose to say this instead. We, as Gricean listeners are licensed to derive the hidden inference because of the form of the utterance.

But does Reebok want us to make the inference that one of those three runners finished fifth? Of course not. The construction “three of the five fastest runners” is interesting for this reason – it doesn’t hide the inference, but it weakens it. Consider the alternatives:

1. Some of the five fastest runners were wearing our shoes.
2. Three of the fastest runners were wearing our shoes.
3. Three of the five fastest runners were wearing our shoes.

(3) seems ideal – it’s technically the truth, and it sounds more impressive than (1) and (2). But I can’t shake the feeling that I’m missing something obvious in my thinking. For one, the partitive construction “Three of the five” should lead to a probabilistically stronger inference as Judith Degen has shown in her work. Also (3) sounds more impressive than (2) right? But why is the inference in (3) non-obvious on a first reading of the marketing slogan? Is there an effect due to the usage of the numeral determiners three and five? Assuming the truth of the statement, is Forty of the fifty fastest runners were wearing our shoes as impressive as (3)?

I’ll need to see what work has been done on these constructions, but sentences of this form seem to illustrate Degen’s non-homogeneity assertion: not all scalar implicatures have the same strength.

Update

Andrew Gelman makes an interesting point in reply to my question of why Reebok would use this phrasing when it could lead to a weaker implicature: people know that something is up but we’re still taken in, similar to how we’re more likely to buy something if its ₹99 rather than ₹100.

Tendulkar became the epitome of values prized in the conventional, hierarchal and self-congratulatory milieu of the middle class, showing no eagerness to challenge the many prejudices of society and state. His notion of ethics remained limited purely to the realm of his own personal conduct … this personal decency has always been accompanied by a deeply ingrained timidity towards authority, a primal fear of upsetting any establishment, whether cricketing or otherwise.

This comment on the middle class resonated with me: respect authority, don’t step out of line, don’t cause trouble, etc. Why risk being anti-authoritarian when you can maintain your place in the social hierarchy by being obedient to authority?

At this critical juncture in my research (and life), I’m glad I found this talk by Maciej Ceglowski. It might seem overtly simplistic, but I appreciate its sentiment, and find it useful to keep in mind when studying bias from a computational perspective:

Instead of relying on algorithms, which we can be accused of manipulating for our benefit, we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It’s a clean, mathematical apparatus that gives the status quo the aura of logical inevitability. The numbers don’t lie.

1. orchard, grove, fruit garden

paradesi in Tamil:

1. Foreigner, stranger … 2. Sojourner, pilgrim, traveller;

What the latter definition leaves out is that paradesi is almost always used as a derogatory word. Are the two words cognates? The Hebrew word’s roots are Proto-Iranian as the etymology describes, while the Tamil word is probably derived from Sanskrit. Proto-Iranian and Sanskrit are both theorized to have developed from Proto-Indo-European. They are probably false friends — true cognates with different meanings due to semantic change.

My interest in these two words began when I learned about the Paradesi Jews of India — is it an accident that a word with negative connotations ties back to people who faced persecution and antisemitism? Maybe, but the sad truth today is that few people in Chennai are aware of its Jewish past. History is the story of people - many stories are tragic but deserve to be remembered.

What were the earliest mass-market PC interfaces like?…they were like conversations…you’d tell it what to do, it gives information back to you…That was the basic paradigm until the Macintosh…What it gave you was not a conversation, but a thing…you could poke the thing and see how it reacts…it worked like a physical object…

I really liked the comparison of early command-prompt user interfaces to conversations. It struck me that today’s AI assistants (Alexa, Google Assistant, Siri) are all based around having conversations. If these systems ever approached anything close to human intelligence and common-sense, perhaps having a conversation is the best way to interact with AI. But I wonder if there is a better interface to interact with AI? What’s the next leap from conversational AI? Perhaps Augmented Reality is the answer — artificial intelligence dispersed in our lived reality, giving us glancable information and the illusion of a physical object we can interact with. I wonder if this is why Apple is so bullish on AR as well.

Or maybe the best way to interact with artificial intelligence is the same way we interact with other people — using conversations.

For me, the answer is no, hence why I’m going to stop using MTurk. Perhaps there are alternatives that are fair to workers, that lead to good wages and job security. Linguistic data is important, and more so when it comes from a diverse population. But I don’t think paying miniscule amounts of money to obtain it is something to be proud of. We can do better.

Update

I’ve been thinking more about this topic after a research seminar discussion – would ensuring that workers get paid atleast $10 per hour fix the problem? It certainly ensures that they get paid a decent wage by you. A few requesters making the decision to pay more(and advertising their decision) isn’t going to make the median wage go up, nor does it do much to fix the numerous other issues with how the platform treats workers. It does help in a small way, and its the least evil thing you can do on the platform.

Update 2

Fair Work is a cool resource from Stanford’s HCI group that runs a server-side script to ensure your crowd workers are paid minimum wage of $15. I found it while reading this excellent opinion piece by Glenn Davis & Klint Kanopka at The Stanford Daily. I agree with their conclusion: its upon Amazon to change the incentive structure – requesters paying workers more and being transparent helps, but from the workers perspective, it will always be a race to the bottom.

One can lie outright about the past. Or one can omit facts which might lead to unacceptable conclusions. Morison does neither. He refuses to lie about Columbus. He does not omit the story of mass murder; indeed he describes it with the harshest word one can use: genocide.

This passage, and especially the last line, made an unusual impression on me when I first read it, and still does. Perhaps it’s the fact that one word needs to do so much work. How can one word possibly convey the magnitude of death, destruction, and suffering? On the other hand, genocide, along with words like racist, do seem to work. Those in power (and the privileged) seem more upset over being labelled racist, or being accused of genocide, than the crimes themselves.

I was reminded of the passage when reading a report describing the forced labour of Uyghurs in China, which used the term — cultural genocide (or ethnocide) to describe the Chinese government’s practises. Words matter, and the crimes against Kashmiris and Dalits need to be called for what they are, regularly and loudly.

Can the decades of military occupation of Kashmir, and the torture and killing of Kashmiris be captured by these words? Or the centuries of discrimination and persecution directed at the Dalit community? Calling these crimes for what they truly are — ongoing genocide, may not be adequate to describe the extent of the horrors that the people have experienced, nor the generational trauma, but it is a good start.

First is the ever more detailed, merely empirical description of the world. This is the nuance of the fine-grain.

Is there a similar phenomenon of nuance rising in Computational Linguistics? Anecdotally, I do see the words nuance and fine-grain more often in new papers and talks. There is a way to verify this — the ACL Anthology provides a bibliography of articles with their titles and abstracts dating back to 1960s. Searching the bibliography using some simple regular expressions, we see that the proportion of articles that mention the words nuance or fine-grain in their titles or abstracts have indeed risen in the last 10 years (there were few mentions before 2000 so I’ve excluded them).

However, the bib file has some issues – older articles are generally missing their abstracts, so we only have their title to search from. Top people are trying to fix the missing abstract field issue, but until then, we can recalculate the proportion of ‘nuanced’ articles by only considering entries that have an abstract field in the bibliography. We observe the same trend:

I don’t think that all of these papers fall into the nuance traps that Healy describes. Nor do I think that this trend of exploring the subtleties and richness of language is bad for computational models and theories of language – my first paper is about exploring nuance within the theory of generics! However, I do fear that I regularly fall into the trap of valuing some research more because it is more nuanced.