IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

GitHub makes Copilot generally available despite weak effectiveness

The AI pair programmer and code-completer has passed a 12-month technical trial but most suggestions are ignored by developers

GitHub Copilot has been officially launched to the wider developer community after passing a 12-month technical preview trial conducted by 1.2 million developers.

The Microsoft-owned company said the AI pair programmer works across “dozens” of development languages and can offer suggestions to auto-complete blocks of code.

Validating concerns from developers, GitHub also said that a recent evaluation showed just 26% of all suggested completions were accepted by developers, meaning that around three in four suggestions are rejected.

Some concerns raised by developers relate to doubts over the accuracy of suggestions, the difficulty required in debugging suggested code, and how much time the feature will actually save.

GitHub said that more than 27% of users’ code files were generated by GitHub Copilot, on average, and the figure rose depending on the language used. The figure jumped to 40% for Python projects, for example. 

“GitHub Copilot does not write perfect code,” the company said. “It is designed to generate the best code possible given the context it has access to, but it doesn’t test the code it suggests so the code may not always work, or even make sense.

“GitHub Copilot can only hold a very limited context, so it may not make use of helpful functions defined elsewhere in your project or even in the same file.”

Copilot may also suggest outdated uses of libraries or languages, due to it being trained on publicly available code, and that suggested code for some languages may be more accurate than others.

Related Resource

Optimising storage infrastructure for DevOps practices

Maintaining IT infrastructure to best support application servicesoptim

Whitepaper cover with text and blue top bannerFree Download

Python, JavaScript, TypeScript, and Go might perform better compared to other programming languages, it said.

The functionality of GitHub Copilot is powered by OpenAI’s Codex generative pre-trained AI model. The model was trained using natural language text, and publicly available code, including some from GitHub projects.

The downside to training such a model on publicly available code is that code samples are becoming increasingly difficult to come by, given the rising use of APIs or new libraries and frameworks.

Copilot will receive ongoing training from new public samples that become available, GitHub said, and the company will provide ways to highlight newer APIs and samples to raise their relevance in GitHub Copilot’s suggestions.

The new product is available now and costs $10 per month or $100 for an annual subscription (£8.17/£81.70) and be used in major integrated developer environments (IDEs) such as Visual Studio and Visual Studio Code, Neovim, JetBrains IDEs through an extension.

The product is free for verified students and maintainers of ‘popular’ open-source projects - whether the project meets the ‘popular’ criteria will be decided on the subscription page. 

All users can redeem a 60-day free trial of GitHub Copilot, too, and the tool will be offered to companies later this year, the company said.

Featured Resources

Accelerating AI modernisation with data infrastructure

Generate business value from your AI initiatives

Free Download

Recommendations for managing AI risks

Integrate your external AI tool findings into your broader security programs

Free Download

Modernise your legacy databases in the cloud

An introduction to cloud databases

Free Download

Powering through to innovation

IT agility drive digital transformation

Free Download

Most Popular

Salaries for the least popular programming languages surge as much as 44%
Development

Salaries for the least popular programming languages surge as much as 44%

23 Jun 2022
The UK's best cities for tech workers in 2022
Business strategy

The UK's best cities for tech workers in 2022

24 Jun 2022
LockBit 2.0 ransomware disguised as PDFs distributed in email attacks
Security

LockBit 2.0 ransomware disguised as PDFs distributed in email attacks

27 Jun 2022