VS Code Commit Message Generation Prompt

In a conversation with Jigar Gosar today we were talking about how VS Code does a decent job of generating git commit messages in the Source Control Sidebar and how he was trying to come up with his own prompt to do it directly from the Copilot chat window.

Since VSCode is open-source and GenAI is great for understanding open-source code, I immediately thought of looking for the prompt used by it. I first went to the VS Code repo and asked Copilot to find the relevant code. Copilot said it couldn't find anything related to AI-generated commit messages and instead asked me to look at integrations with GitHub Copilot. Since the VS Code Copilot Chat was also open-sourced a few days ago, I went to it's repo and again asked Copilot to search and it quickly pointed me to the right place to look for it. 

So here is a snippet of the prompt used to generating the commit message in VS Code: 

You are an AI programming assistant, helping a software developer to come with the best git commit message for their code changes.
You excel in interpreting the purpose behind code changes to craft succinct, clear commit messages that adhere to the repository's guidelines


# First, think step-by-step:
1. Analyze the CODE CHANGES thoroughly to understand what's been modified.
2. Use the ORIGINAL CODE to understand the context of the CODE CHANGES. Use the line numbers to map the CODE CHANGES to the ORIGINAL CODE.
3. Identify the purpose of the changes to answer the *why* for the commit messages, also considering the optionally provided RECENT USER COMMITS.
4. Review the provided RECENT REPOSITORY COMMITS to identify established commit message conventions. Focus on the format and style, ignoring commit-specific details like refs, tags, and authors.
5. Generate a thoughtful and succinct commit message for the given CODE CHANGES. It MUST follow the the established writing conventions.
6. Remove any meta information like issue references, tags, or author names from the commit message. The developer will add them.
7. Now only show your message, wrapped with a single markdown ```text codeblock! Do not provide any explanations or details

Further:

Now generate a commit messages that describe the CODE CHANGES.
DO NOT COPY commits from RECENT COMMITS, but it as reference for the commit style.
ONLY return a single markdown code block, NO OTHER PROSE!

You can find this in the following file:

src/extension/prompts/node/git/gitCommitMessagePrompt.tsx

A quick scan of the code also shows that you can provide your own custom instructions for generating the commit message.

GenAI can make a big difference in understanding and working with legacy code and contributing to open-source.

The GenAI Software Development Spectrum

In my usage and conversations with other "builders", here is the spectrum of GenAI usage I've come across.  

L1. Ask/Consult - AI as a Knowledge Resource

Use AI as a replacement for reading manuals or documentation, and using Google or Stack Overflow

• Manual/Documentation Lookup
• Explain/Understand Code
• Ask about Best Practices and Options

L2. Augment - AI as a Pair Programmer

Work together with the AI in small iterative increments where you bounce off ideas and take turns where you do some of the work and the AI does some of the work.

• Auto Complete
• Debugging
• Refactoring
• Code Quality Review
• Optimisation and Performance Review
• Security Review

L3. Delegate - AI as a Junior Programmer

After some planning and discussion with the AI, give it tasks and have it write the code that you review before accepting the pull request. 

• Specs, Requirements, Plan with Tasks
• Assign Tasks to AI to writes code
• Review pull request for the code written by AI

L4. Outsource - AI as an Outsourced Freelancer or Agency

You only care about the outward functionality of the software and you outsource the engineering/development to the AI.

• Come up with the Spec and Requirements
• Tell the AI to come with a plan and tasks
• Ask the AI to "build" the software.
• Test the functionality of the software as a user

Questions for the Reader

1. How do you use GenAI?
2. Do you have a different model for thinking about this?
3. Have you come across other ways of GenAI usage in software development?

GenAI is great for Understanding Legacy Code and Open Source Contribution

While GenAI can be a high speed legacy code water hose, that can be used to create tons of legacy code instantly, it's also a great tool for understanding legacy codebases and orienting yourself.  It can also be used to build ephemeral test scaffolding to get a safety net before making changes to the legacy code. 

This, IMO, is not just great for managing legacy code, it's also great for open source software. You can use GenAI to quickly understand the codebase "just enough" to make the contributions you want to make.

The GenAI Product Management Traps

Feature Factory / Kitchen Sink

GenAI greatly reduces the barrier to directly and quickly build features which makes it very easy to act as a feature factory and build a kitchen sink of a product. One that is not true to a design or have an opinionated view.

• When implementation is cheap, it's easy to end up in a mess.
• When you try to make everyone happy, you end up not making anyone happy.

Big Design Upfront / Waterfall Software Development

In an attempt to prevent GenAI from making mistakes there is a focus on front loading effort in the specs and requirements. 

• We need to iterate and have to discover what we are building
• Action precedes clarity
• Account for emergence

All these issues boil down to the question of Are We Building the Right Product?

GenAI Multi-Pass

Given the non-deterministic nature of GenAI and a near future where software development might not involve opening an IDE to write software, one way to catch issues might be to take a multi-pass approach where each pass is specialised for a specific purpose, so as to not overload the LLM with too many things to take care of at one time. Some of these multi-passes could be:

• Coding Pass
• Code Quality Pass
• Security Pass
• Performance Pass

Each of these passes could result in improvements tasks and this could be an iterative process till a certain threshold is met.

The Excitement and Frustration of using GenAI for Software Developement

Exciting

• The possibilities that GenAI enables makes software development truly exciting again.

Addictive

• It's so easy to convert ideas to working software, it can be very addictive and you can quickly loose any sense of time. 
• It keeps you from wasting time, being unproductive or procrastinating

Frustrating

• The non-deterministic nature of GenAI can make it very frustrating.

Time Sink

• It's very easy to get sucked into long conversations, and issues specific to GenAI, than spending time actually building.