We Tried 3 AI Commit Tools – Here's Our Honest Review

With the ongoing advances in Artificial Intelligence (AI) and its increasing integration into developer toolkits, committing code has never been easier. Today, we’ll dive into an evaluation of three new AI-powered commit tools: OpenCommit, aicommits and GPT-Commit Summarizer.

1. OpenCommit

OpenCommit was the winner of the GitHub 2023 Hackathon, and here is why.

Features:

• CLI and GitHub Action Integration: OpenCommit offers both a Command Line Interface (CLI) tool and a GitHub Action. This flexibility allows users to commit changes either manually through the terminal or automate their commit message enhancement using GitHub Actions.

• Customization: It provides several configuration options, from emojis to detailed commit descriptions. It even supports GitMoji, a fun and informative way of prefixing commit messages.

• Locale Configuration: For global teams, OpenCommit supports multiple languages, making it more versatile across different developer communities.

• Git Hooks: This feature is indeed a killer. By integrating directly into the Git workflow, OpenCommit can automatically generate commit messages without the developer needing to manually intervene.

Ease of Use:

Setting up OpenCommit is straightforward. 

After a global npm installation, users need to get an API key from OpenAI. Once done, they can run OpenCommit using simple commands like oco.

Pricing:

OpenCommit leverages the ChatGPT (3.5-turbo) model, which is considerably cheaper than GPT-4. However, be prepared for the costs associated with OpenAI API requests.

Review:

OpenCommit is impressive, both in its feature set and its execution. Its deep integration with Git makes it a favorite, especially for those familiar with the Git workflow. The only potential drawback is the cost associated with frequent OpenAI API calls.

2. Aicommits

With over 1.7k stars on GitHub, aicommits offers a different flavor of AI-powered commit messages.

Features:

• Simplicity: aicommits is primarily a CLI tool. It retrieves your code changes, sends them to OpenAI's GPT-3, and produces a commit message.

• Future Plans: The roadmap for aicommits includes support for conventional commits, the ability to alter generated commit messages, solutions for latency issues, and more.

Ease of Use:

Installation and setup are uncomplicated. Like OpenCommit, users need an API key from OpenAI. The straightforward aicommits command will then generate commit messages for any repo.

Review:

aicommits is for those who cherish simplicity. While it may not have the extensive feature set of OpenCommit, it does its job well. We're excited to see its planned features roll out, potentially making it a stronger competitor in the AI commit tool market.

3. GPT-Commit-Summarizer

At its core, GPT-Commit-Summarizer is a GitHub action. It leverages the text-davinci-003 model (known for powering ChatGPT) to generate concise summaries for pull requests. But its brilliance doesn't stop there. It also furnishes descriptions of the changes made to individual files and specific commits.

Setting It Up

The setup is remarkably seamless. A user only needs an OpenAI API token and a single workflow file added to the repository. This ease of integration, combined with its universal compatibility with all programming languages and frameworks, certainly works in its favor.

Functionality in Action

Our testing was extensive. We pulled data from various real-world codebases, including personal projects and notable names like ditto.fit and connectedpapers.com. The results were intriguing:

• Pull Request Summaries: The "PR Summary so far" comments were a standout feature. They're generated incrementally, ensuring the discussion page isn't swamped with repeated comments if a commit is added. This incremental approach serves as a boon, enabling reviewers to grasp the essence of a pull request swiftly.

• File Diff Summaries: These are usually precise and are instrumental during reviews. If any change baffles you, a quick look at the summary of the commit where it transpired typically provides clarity.

• Maintenance: The tool is self-sustaining. It auto-deletes outdated summary comments, ensuring that the review page remains clutter-free.

To comprehend its inner workings, understanding the limitations of the model it employs is essential. With a restriction of 4096 tokens, it can handle about 200 lines of code. This might seem meagre compared to a sizable pull request.

For a deeper understanding of AI-driven code interpretation, check out Hatica's insightful article on ChatGPT's code interpreter feature.

What about GitHub Copilot Commit?

While GitHub Copilot's PR feature is still under development and thus not under our review, the glimpse we have so far is intriguing. With features like AI-generated pull request descriptions and the ability to expand markers into detailed lists of changes or even poems, Copilot is pushing the boundaries of what's possible with AI in development. The upcoming features, such as Gentest and Ghost Text, further emphasize Copilot's potential to be a game-changer.

Conclusion:

OpenCommit impresses with its feature-rich approach, seamlessly integrating with Git. aicommits offers an elegant, minimalist touch that does not compromise on quality, with promising updates on the horizon. GPT-Commit-Summarizer simplifies and enriches the review process with its concise summaries. Meanwhile, GitHub Copilot teases an even brighter future for AI in development.

While no tool is perfect, and results might vary based on the complexity of the codebase, the potential it brings to the table is undeniable.