Express Forward

[wesleytodd]

Hey Everyone! I hope this issue finds you doing well, it has been a few years since I last posted in the project but it has been on my mind for a while that I wanted to do this. Thanks to some great help from @sheplu and @UlisesGascon (triage team members and folks active in the larger Node.js Ecosystem) we wanted to put forward a plan for the future of the Express project.

As we all know, the project has been in more of a maintenance mode for a long time now. Since the 5.x branch has not shipped despite a concrete plan to do so in 2020 we believe that in order to prevent the ecosystem from having to deal with more drastic measures, that we should make an effort to revive the project starting with a renewed look at the governance to help bring new contributors into the project.

Ideally this plan is uncontroversial and can be quickly acted upon. To do that, we thought that we should schedule a TC meeting with at least the folks last listed as active TC members (@dougwilson @LinusU @blakeembrey @crandmck @wesleytodd) and interested community members to discuss and commit to a direction. So below is the plan we worked out to get us back onto a healthy track, after kicking off this discussion here I would like to open an issue to schedule a TC meeting for some time in the next few weeks.

Plan

The plan comes in phases and focuses on direct tactical steps as opposed to strategic goals. Despite this, most of the tactical goals are backed up by larger strategic goals which we are omitting for brevity. Additionally there are some todo's and areas of ambiguity here. Ideally these would be filled in by collaborators as we move along and should not be blockers to agreeing on the general direction and goals.

Immediate Needs

These are things that we propose be done immediately upon approval of this plan.

• Review and Adopt the Repo Captains Proposal
  • Review and make any last changes to wording, then merge
  • Immediately give commit access
  • Immediately give publish rights on npm
  • Setup a fast follow for onboarding more repo captains
• Reform Active TC and open up new TC seats
  • Goal of 5 active members at all times going forward
• Add Foundation accounts to all orgs, repos and npm packages

Express 5.0

• Publish bit on all packages shared with the Foundation
  • Release ASAP
    • Last update thread
    • Last PR list from 2020
    • Unless we meet and find glaring issues, we should release immediately
• Address any remaining issues in patch/minor release as a fast follow
• Change Express-Generator to use express 5 as a default
• Blog and announcements
  • This should be shown and shared to have the community aware of the changes and need to update

Express 6.0

• Evaluate Node.js version support
  • Better align with Node.js LTS schedules
  • This helps accelerate the ability for Express and Node.js to move forward
  • It helps folks have a more predictable upgrade schedule for breaking changes
  • It helps plan for when folks need to land breaking changes on Express
  • This doesn’t mean we would make breaking changes for the sake of it, just that we wouldn’t test or explicitly support non-LTS versions on new major releases made past their EOL
• Automated release process
  • Move release auth into GH permissions
  • Automate releases across all repositories
• Remove unused middleware
  • TODO: assemble list
• Replace middleware by node.js core solution
  • TODO: assemble list
• Remove template and rendering from Express
  • Move to middleware
  • Express as an API first class citizen
• Restructure Github orgs to clarify ownership and focus
  • Go through each and decide a fate
    • https://github.com/orgs/jshttp/repositories?type=all
    • https://github.com/orgs/pillarjs/repositories?type=all
  • Deprecate actually inactive repos, then archive
• @express scope on npm for sub packages
  • All new packages under the scope
  • Asses if we want to move any existing packages
• Stop using Node.js internals / monkey-patching Node.js
• Officially maintained TypeScript types
  • Generated and tested, with docs
• Express Generator revamp
  • Target for API only (remove templating for HTML render`)

Express 7.0

• Ecosystem collaboration
  • Work with node core on future for many of the express owned modules
  • Web Platform things like Request/Response and URL
  • Rewrite middleware to be added to node core / rewrite for performance
  • Collaborate in the Web Server Frameworks Team
  • Stop using Node.js internals / monkey-patching Node.js
• LTS Plan and Release schedule
  • Mirror Node.js LTS and setup major release schedule
• Modernization effort
  • Directly support meta frameworks, Graphql, and GRPC style usage
  • New website

All of this is up for discussion, the goal here is more about rallying people together to help progress happen but we thought coming with a concrete list of items would be more productive so please feel free to discuss individual points. If we need to we can break the discussion up but use this issue as a hub. And ideally the first order of business is getting the TC meeting scheduled and Express 5 out the door, so let's not let perfect be the enemy of the good here and commit to starting and keeping the progress flowing.

[dougwilson]

👏 thank you so much @wesleytodd for this!

[crandmck]

I've been MIA for quite some time, but I'd love to see this project pick up some steam again.

I will try to attend a TC meeting if I am able. LMK if there are candidate proposed times. My schedule is often full during PT business hours, so 5pm-9pm PT is likely to work better for me, but I will try to make any time that is chosen.

[wesleytodd]

I created an issue to get a meeting organized: #161

[broofa]

As a bit of an outsider (albeit one who's been using express for over a decade), the only part of this I find of real interest is how you plan on changing the organization and development processes to allow for regular releases.

I say this because Express has needed a new major release for at least 7 years, since async-await dropped in node@7 (circa 2017). V5 should have been that overhaul but it's just... never... dropped. The reasons for that are important because they're still pretty clearly in evidence.

Until these issues are addressed I question how much value there is in discussing a v5 release, let alone v6 or v7.

If I had to guess at the root cause it's that with a community as large as Express', dropping a new major release will involve a significant amount of follow-on work: issues to investigate, PRs to review, middleware projects to help upgrade, etc. But none of the current maintainers have time for that.

That makes for a certain lack of interest in actually clicking the "Publish" button.

Anyhow, that's my high-level take on this. With that in mind, some suggestions ...

💡 I like the idea of onboarding new team captains and TC members. I'd suggest making this the primary focus for the foreseeable future. This project needs new, fresh faces to help carry it forward.

💡 I like the expectation that team captains and TC members be "active". Do not underestimate how much passive team members can drag a project down simply by doing nothing. I'd make a point of starting the TC meetings with an informal and supportive poll to get a sense of everyone's level of availability and involvement.

😱 Awkward aside: The existing TC has not met for 3+ years. It has allowed v5 to go unpublished for 8-9 years. Is there a definition of "active" that is consistent with this? If not, what does that imply...?

💡 Does the TC have a chairperson? If not, it should. This new "push" is a non-trivial effort. It will require someone designated with spearheading it, and to insure the new TC and process(es) that get put in place stick. Whoever steps into this role should not do so lightly. It will be a significant amount of work.

💡 Use the v5 release to vet the new team structure, especially when it comes to release and support resources.

• Have a "captain" for CI and release tooling.
• Have a "captain" for support
• Have a "captain" for media and community engagement

💡 I would also look at adding support for sponsorships. Recruiting maintainers is one of the hardest parts of running an opensource project. Providing a means of compensation will help with that.

My apologies if this ruffles anyone's feathers. If I'm misinformed or ill-informed, please do let me know. It'd be great to see this project move forward and continue to evolve.

[dougwilson]

@broofa you should consider joining ❤️ 😃

[wesleytodd]

the only part of this I find of real interest is how you plan on changing the organization and development processes to allow for regular releases.

Until these issues are addressed I question how much value there is in discussing a v5 release, let alone v6 or v7.

That is explicitly why the "immediate needs" are what they are! Getting the right structures in place to operate the project in a more healthy way as nearly always been my goal since I started participating in the project and I think the support in this issue shows that maybe we are aligned on that goal.

If I had to guess at the root cause, it's that with a community as large as Express' dropping a new major release will involve a significant amount of follow-on work: issues to investigate, PRs to review, middleware projects to help upgrade, etc. But none of the current maintainers have time for that.

We have had no lack of volunteers over the years. What we didn't have was the structure in place to foster them and get them to be strong active contributors. This is what I hope to fix by opening this issue and calling for these things to be addressed. The reason for even including the concrete list of steps for 5/6/7 was to put some clarity behind this that that work is required.

The existing TC has not met for 3+ years. It has allowed v5 to go unpublished for 8-9 years. Is there a definition of "active" that is consistent with this? If not, what does that imply...?

I don't want to get too in the weeds here, but personally I stepped back because lack of progress caused me to burn out. If we cannot get the governance in place with a truly active group in the near future then I will personally be stepping down from the TC. We have verbage for moving members from active to inactive in the charter, but it is not enough. This is part of what I meant we needed to address under the "reform the tc with 5 active members".

Does the TC have a chairperson?

I tend to agree we should have this, but we would likely need to ammend the charter to make that happen. I am on board if we want to add that to the agenda.

I would also look at adding support for sponsorships

I have opened issues for this in the past in this repo. I agree we should do this. Lets figure out how to slot those into the plan.

My apologies if this ruffles anyone's feathers.

Not at all!!! The whole point of this was to draw out folks like yourself who had good ideas on how to unstick the project. Thanks for taking the time to write this all up and please think about coming to our meeting to help us achieve these goals.

[broofa]

please think about coming to our meeting

Of course. If nothing else, it's a chance to put some faces and voices to names I've interacted with numerous times over the years. (No promises about getting involved, though! 😉 )

[dougwilson]

I agree with @wesleytodd said above. We probably need to have a complete, cold hand over to others and hope they can get up and running. I just no longer have the time or energy to carry such a large project, just code wise, let alone anything else. As I mentioned to @wesleytodd for the past, probably couple years the endless flow of flase vulnerability reports and threads of filing CVEs, trying to argue with people, many times have never even used javascript, just sucked away all my time. If I didn't tend to it, probably every module in express would be security blacklisted for no reason. I am honestly kinda done with all this non sense Express seems to attract, and of course we now have the endless SPAM PRs on the main repo I cannot get to stop. Some awesome folks really would be awesome to step in and figure this all out 🙏

[wesleytodd]

and hope

Other than this part, I agree. I think we need to do more than hope and I am personally volunteering to help make sure we do more than hope. I have time to help, just not the time or will to do it alone or with a group too small to achieve the goals. Luckily I don't think we will lack volunteers if we have the right setup for them to succeede.

[dougwilson]

Haha, fair. I pretty much only have at best 1 hour a day to work on any OSS any more, so hopefully that puts into perspective how constrained I have been. I feel like @wesleytodd hinted at, I think the best thing we can do for the project is pretty much an accounting of all the stuff, get everything moved to the foundation accounts and stuff, and I can disappear into the night 😃 bc honestly I probably need like a year break from this OSS stuff, like a sabbatical.

Edit: and I don't want my very limited availability to stop whatever momentum that comes from all this!

[wesleytodd]

Burnout is REAL, and OSS can create an environment where it is easy to hit. I think we can all say we really really appreciate all the work you have done to keep things afloat. And now hopefully we can get a group of folks who can help take that off your shoulders (or help you find a better future balance with the project after you take that well deserved break)!

[dwelch2344]

Hey all! Just wanted to throw my hat in the ring here: would love to help contribute where I can. Also want to look at sponsorship, along with how our team could commit cycles on specific efforts!

I'm going to bring this up internally with our team tomorrow, but personally I'd love to get started on the CI/CD front. I haven't dug into the repo yet to see what's in place, but I'm a huge fan of Commit -> GH Actions -> Release pipelines as a starting point – and given some of the items above it sounds like that could be of use?

I'm not sure I could commit to being a captain or anything, but I'd love to be workhorse if someone wants to point me in the right direction. Docs, tests, builds, whatever :)

[ljharb]

I'd also love to help out; I don't have time to write code, but I'm happy to join the TC or be a repo captain, or anything similar that would help.

[wesleytodd]

Thanks @dwelch2344 and @ljharb! We absolutely need folks in all sorts of capacities to deliver on all the things on the list above, so everything from setting up and maintaining better shared CICD workflows for the many repos to helping provide technical leadership and direction for the community will be very welcome! If you haven't yet, check out #161 and add your intent to attend (with your TZ) so we can find a time which works for most people.

After that meeting we will likely have to revive a few issues and reach alignment on the way forward before we start pointing folks in direction for work, but hopefully that shouldn't take long once we get started.