Comparing Contrasting AI Policies: from IISER Pune to Zig

IISER Pune has recently announced an AI use policy laying out the conditions under which AI use is permitted (encouraged?) in the institute. The open source software project Zig has announced a policy that goes completely in the opposite direction—banning all contributions that have used AI. And I found both of them interesting (and useful?) for different reasons.

AI is changing the world in good ways and bad. We need to find a way to allow and encourage the good ways while at the same time prohibiting or discouraging the bad. This is a tricky balance, because getting the policies right is difficult and getting them wrong will be very dangerous, but not having policies might also be dangerous. So everybody needs to deal with this: countries, companies, educational institutions, and parents.

For example, a country that prematurely regulates AI runs the risk of discouraging experimentation and innovation in AI, and could quickly fall behind other companies. But without regulation, maybe various serious problems caused by hallucinations (for example, in hospitals, courts, schools and colleges, etc), or hacks and scams, or worse.

Companies struggle with hallucinations and privacy issues vs being left behind.

Educational institutions worry about cheating and cognitive offloading (and the resulting cognitive decline) vs not preparing the students for an AI future, and essentially operating with one hand tied behind their backs.

So every attempt at a country or institution announcing AI policies is an adventurer exploring uncharted territory and taking risks so that we all can learn from their experience. Some such policies are questionable and result in quick harms, but others seem sensible and worth following closely.

Last month, IISER Pune, arguably India’s top undergraduate science institute, announced its AI Policy. Here are a few points that stood out for me:

1. Use of Generative AI is permitted in all submissions and activity of the students and staff unless explicitly forbidden by the instructor1. This includes reports, assignments, thesis, and articles formally submitted either as part of a course or an academic program.

2. If an instructor has prohibited the use of AI, they have to clearly indicate how they will enforce this. The policy specifically points out that the use of automated software to detect whether something is AI-generated is not reliable, and hence should not be used. So the instructor should let the students know beforehand how this will be verified (for example, an in-person viva). Instructors are allowed to just trust the students (honor system).

There are a number of other interesting points in the policy, including mandatory attribution of AI use. I recommend reading the full section 4 of the document (just 5 pages).

I am (was?) a big fan of take-home exams and open-book exams, but I’m afraid those have been rendered completely useless by AI. In-person exams, or take-home exams followed by in-person vivas will be the primary methods for evaluation of students. Here is a good middle ground by Kaneenika Sinha (Associate Prof at IISER Pune, and a member of the committee that drafted the above policy):

In general, I like IISER Pune’s policy because it is an AI-friendly policy, which I think is far better than the approach being taken by many educational institutes, which is to ban AI use, and enforce it using AI-detection software (that all students know how to bypass trivially).

Zig, a major open source software project (it is a relatively new programming language), has recently announced its stringently anti-LLM AI policy:

Strict No LLM / No AI Policy

No LLMs for issues.

No LLMs for pull requests.

No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words.

Why, you might wonder, how do I find this policy interesting, considering that I’m solidly in the pro-AI camp so far?

This is the current situation with open source projects:

1. Anyone can contribute code, bug fixes, and modifications to open source software projects. These are called PRs (for pull requests2).

2. The core-contributors of the project (i.e. the people in charge, usually, but not necesssarily, the ones who started the project) have to look at all the contributions and decide which ones are good enough to accept in the project, while others are rejected.

3. For large, popular projects, the “anyone” literally means “anyone”. So such projects get lots of contributions that are low quality, and in many cases, downright terrible. Picking out the good contributions from the mass of bad ones is an ongoing challenge for such projects.

4. In the last few years, the number of such contributions has increased exponentially, because of AI use.

5. The AI-enabled contributions are not necessarily bad. Some of the AI contributions are slop, but many are decent.

6. The normal policy usually is to accept all good contributions and reject all bad contributions, irrespective of whether the contribution was pure-human or AI-assisted or fully AI-generated

7. But Zig has chosen otherwise

In a post titled “Contributor Poker and Zig's AI Ban”, Zig’s VP of Community, Loris Cro explains:

it would make sense to stop accepting imperfect PRs in order to maximize ROI from your work, but that’s not what we do in the Zig project. Instead, we try our best to help new contributors to get their work in, even if they need some help getting there

Simon Willison explains it as:

Zig values contributors over their contributions. Each contributor represents an investment by the Zig core team - the primary goal of reviewing and accepting PRs isn't to land new code, it's to help grow new contributors who can become trusted and prolific over time.

Loris Cro puts it as:

The reason I call it “contributor poker” is because, just like people say about the actual card game, “you play the person, not the cards”. In contributor poker, you bet on the contributor, not on the contents of their first PR.

Sports fans would know this as “play the man not the ball”. People in the startup world would know this as “VCs invest in the team not the idea”.

Zig makes an important point. One potential problem with increasing AI use is that now seniors in most industries can use AI to do a lot of the low-level grunt work that they earlier used their juniors and interns for. This is great for the seniors: it is cheaper, much lower management hassles, much faster, and the AIs are significantly more compliant. The problem is that this is how the juniors used to learn enough on the job to become experienced seniors. Now, where will the seniors come from. AI is hollowing out the AI pipeline.

I don’t know if Zig’s approach will help with this problem, or whether their approach is even sustainable. But I think nobody knows how to solve this problem so there should be lots of different people experimenting with lots of different ways of doing this.

Specifically, experimentation with explicitly stated policies is needed, otherwise in both the cases above, the humans involved (instructors in educational institutions, and seniors in industry) will take the easy way which will probably not be the right way.

I don’t know the answer to the questions raised. But this is something we all have to think about; most of us aren’t in a position to shape the policies of a country or a large organization, but some of us can shape policies in smaller organizations, and we definitely want policies in our homes, and our children. So we should keep track of the developments in this area, and we should be open to experimentation using the “strong opinions loosely held” principle