It's been three months, what happened in pip?

As I announced in a previous post, the pip project secured funding to have me work on pip on a part-time basis from June to August this year.

The contract period is now finished, so it’s time to look at what was delivered, what work remains, and the reflections I have.

Beforehand, though, I’d like to express my greatest apprecation for the Python Software Foundation, the Packaging Workgroup, and my pip co-maintainers for their trust and this opportunity! It has been an opportunity like no other.

What was delivered

Virtual environment build isolation

This was my first work item, and I’m happy to say that --use-feature=venv-isolation was shipped in pip 26.2. The feature was quite a bit more complicated than I was expecting – especially since I did have a working prototype for years – but I’m so glad it exists now.

Python 3.15 broke our current build isolation mechanism, and the fix we implemented has had unintended consequences. It’s getting increasingly difficult to patch all of the ways our heavy-handed isolation mechanism behaves poorly, so I’m looking forward to replacing it with venv-isolation.

In particular, it feels good that --use-feature=venv-isolation is a solution for the linked issue.

Deprecation of legacy resolver

Some, albeit limited progress has been made on fully deprecating the legacy resolver.

Pradyun’s fixes for a handful of long-standing bugs of the resolvelib resolver were included in pip 26.2 (without issue). Furthermore, I have prepared language to use when we will be communicating the legacy resolver removal more broadly.

The last blocker is whether and how to support Direct URLs dependencies satisfying editable requirements to equivalent locations. There have been four related PRs (#1 #2 #3 #4), and work is still on-going as the feature may possibly expand to prioritizing editable requirements over equivalent direct URL requirements generally and supporting editable constraints.

These features are heavily intertwined and involve nontrivial changes to the resolver, so I expect these to take a while.

The upside is that I did familiarize myself with pip’s resolver code (which was a secondary goal). I feel more ready to take on the removal of the legacy resolver whenever it does occur.

Error improvements

In June, I shipped new diagnostic errors for HTTP connection errors that are easier to read and provide useful advice to the user.

Later, I took a brief detour to fix an uncaught exception when the build backend failed to be imported. Afterwards, I soon filed a draft PR to rewrite urllib3 retry warnings to be understandable after reaching a compromise with the urllib3 maintainers. Once upstream changes to urllib3 are released, the HTTP retry warning will be concise and informative.

$ pip install https://expired.badssl.com --retries 3 --timeout 3

Collecting https://expired.badssl.com
  WARNING: SSL verification failed while connecting to expired.badssl.com, retrying 3 more times
  WARNING: SSL verification failed while connecting to expired.badssl.com, retrying 2 more times
  WARNING: SSL verification failed while connecting to expired.badssl.com, retrying 1 last time
error: ssl-verification-failed

× Failed to establish a secure connection to expired.badssl.com while fetching https://expired.badssl.com/
╰─> [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1081)

hint: You may need to use --cert or check your proxy/firewall configuration

Near the end, I filed a PR for improving “this wheel is incompatible with this platform” errors.

$ pip install numpy-2.5.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl --dry-run

error: incompatible-wheel

× numpy-2.5.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl is incompatible
╰─> Wheel architecture is unsupported: aarch64 (current: x86_64)

hint: Run 'pip debug -v' for a list of compatible tags for your system.

Index priority (or not)

I ended up removing index priority as one of my working items. I left a detailed explanation on why in one of my updates, but the TL;DR is that index priority is a huge feature and minefield. I was hoping that there would be a viable way to implement a simple form of index priority that would be beneficial, but after further discussions, there was not.

In light of that, I decided to instead work on improving and ensuring full compatibility of --use-feature=inprocess-build-deps with the venv-isolation. In-process build dependencies is an experimental feature that I shipped earlier in the year (before this grant work). It works great, and like venv-isolation, will fix many long-standing issues. However, it was stuck in limbo since IMO it wasn’t quite ready to be enabled by default (especially after realizing it was mildly incompatible with venv-isolation). In interest of avoiding features being stuck in experimental limbo forever, I decided improving inprocess-build-deps was the next best thing to do.

I know this is disappointing since index priority is one of the features that many people would welcome and appreciate. I’m sorry. It’s just that it’s better to get index priority right over rushing.

General maintenance

I’ve made good progress here, having decided that focusing on code review was the best use of my time.¹ In total, I reviewed 93 pull requests, resulting in pip 26.2 being a massive release with many features and bugfixes.

I’m also happy to report that the open PR backlog sits at around 90, down from 130 when I first started working in June.

Whenever I got bored, I closed stale or already resolved issues in pip’s issue tracker.²

Overall, the net change in open issues since end of May is around -92. 🎉

Our new triagers, Sepehr Rasouli and Youngkwang Yang, have done the majority of the leg work here, however. Without them, the drop in the past three months would’ve been far more muted.

Finally, on the note of new triagers, I’ve spent some time during these months reviewing and providing support to our newest members of the pip team. While this wasn’t a part of the original goals, I fully believe investing in future contributors. It’s the only way pip can be maintained for the long-term.

What work remains

• Further work on improving inprocess-build-deps and compatibility between it and venv-isolation. A PR to replace our spinners is the first part. I have a work-in-progress PR to address bugs and limitations of the in-process build dependency installer, and I’m planning to follow up with more tests.

• Fixing the last technical blocker for the deprecation of the legacy resolver and scheduling a removal date. This is likely to occur on a timescale of quarters, contingent on when a pip maintainer (probably me) has significant free time to volunteer.

• Further work on improving “no distribution found” resolver errors. I did take a look here, even prototyping an improvement when requires-python prevents a distribution from being found, but I quickly realized that making improvements here is highly nontrivial.³ For now, I plan on shipping small incremental improvements where possible.

• Reviewing and shepherding the index error improvement and invalid distribution metadata PRs that triager Sepehr Rasouli has submitted. They are nontrivial changes, but they offer a meaningful improvement to UX.

• Addressing feedback on and landing my current open PRs :)

Reflections

• I was way overly optimistic with my assumptions on how much work I could accomplish. In hindsight, I should’ve scaled down my expectations. Underpromise and overdeliver, really. For what it’s worth, this is my first real job in the software industry, so this is a professional lesson learned.⁴

• There is never enough time for PR review. I spent a huge portion of my time reviewing PRs in the month or so before release 26.2, and yet it feels like I hardly made a dent. Regardless, it was extremely beneficial to have dedicated time for review because it’s often the most time consuming and difficult part of being a maintainer. Not only do you need the technical skill and codebase expertise, but doing code review right is also a social skill, especially when giving feedback to regular contributors.

  • This was also an issue for me since I’ve been waiting for some time to have my work reviewed (or in some instances, not reviewed). There were intentions to have more maintainer time for review during this period, but through the happenstances of life, that time never materialized.

• OSS maintenance is a job in itself. Continuing from the previous point, OSS maintenance is a lot of work. It’s not even the amount of work, but the type of work if you want to do it “properly”. There are simply parts of the work that I don’t enjoy (as much) or require lots of social/mental energy, of which I have in limited quantities. I’ve known this for many years, having maintained Black in a previous life, but I didn’t internalise just how much of a job it really is until I was paid for this work.