Deprecate partial functions from the Prelude, giving them new quarantined homes

[Ericson2314]

Problem

Partial functions like head are too easy to use. The purpose of the Prelude is to guide the user, to make certain things very easy in order to encourage Haskell to be written in a certain way. The problem is that the partial functions are bad, as is widely agreed. It is unclear if there are any circumstances in which as they ought to be used, and yet Prelude makes them extremely easy to use at present, especially by new users that don't know what pattern matching is.

Many would-be solutions have been floated in the past, but I dislike them as tepid half-measures that skirt around the problem rather than confront it head on. The problem is that these partial functions are too readily available; the solution must be to make them less readily available. The solution is to make using those in the Prelude automatically raise red flags, and ideally to someday remove them from the Prelude altogether.

Solution

We want to move these functions with a very long deprecation cycle.

Prerequisite

https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0134-deprecating-exports-proposal.rst must be implemented first.

Steps

1. Relocate functions, with reexports

This we can do immediately!

• Create Partial modules and move functions there:

  • Data.List.head -> Data.List.Partial.head
  • Data.List.tail -> Data.List.Partial.tail
  • Data.Maybe.fromJust -> Data.Maybe.Partial.fromJust
  • ...TODO fill out more

• Create non-deprecated reexports (since we lack the ability to do anything better) in their current locations (Data.List, Data.Maybe, etc.).

The purpose of this is allowing one to switch sooner once the deprecation mechanism is ready without running afoul of the 3 release cycle.

2. Deprecate Reexports

This we can do once the GHC proposal is implemented.

• Deprecate the reexports of these function in their Prelude reexports those and thus inherits the deprecation.

3. (Non-normative) After many releases, and good tooling, consider removing reexports

We're not sure what is needed to do this, so just including as an aspirational goal / food for thought.
At the very least, this could only happen after:

• 3 release policy is met, so we must have both 3 past GHCs and bases which have these functions available from the the Partial modules. (probably would be far more than 3 releases.)

• Tooling exists to automatically perform migration guide

• Perhaps also some sort of "base shim" so code explicitly depending on the old standard library (as opposed to say Haskell just dropped in GHCi without metadata) never stops working.

It is important to remember that not everyone that signs up to publish open source wants to be stuck maintaining it for life. Some people want to make a thing, release it, and wash their hands of it ---- think e.g. artists aren't expected to accept "patches" against a finished painting! Any hard breakage, no matter the migration tooling around it is going to to result in people pestering those once and done authors who will rightly ask "why is stuff in the language report for crying out loud changing out from under on me?"

For that use-case, only something like the base-shim that requires no changes in existing code --- essentially making this not a breakage change but just changing the defaults for new projects --- is acceptable. Indeed a social contract that a) we can and will retroactively add more and more warnings to old standards b) we will never change the meaning of old standards, is probably a good way to navigate these competing concerns in general.

Migration Guide

Merely import and ...Migration module that provides an in-use function gotten from a legacy deprecated location.

Note these properties:

• Qualification is not necessary. One may choose to do e.g. import qualified Data.List.Partial as Partial, but import Data.List.Partial is fine.
• Manual Prelude import is not necessary. One may choose to do e.g. import Prelude hiding (head), but this line can be omitted entirely.
• Import lists are not necessary. One may choose to do e.g. import Data.List.Partial (head), but import Data.List.Partial is fine.

These properties follow from the nice benefits deprecated reexports provide, benefits we do not have with today's means of doing a deprecation cycle.

[tomjaguarpaw]

Strongly in favour of attempting to build community consensus around this.

[Profpatsch]

Very much in favor of only doing breaking changes to base with a migration tool. We should have the technology!

[andreasabel]

Concerning partial functions likehead, tail, !!, length etc. on lists, they stem from the conflation of three type-theoretically different concepts:

1. finite lists (List a)
2. possibly infinite lists (CoList a)
3. streams (Stream a) which are infinite sequences

These three are lumped together in a single type [a].

• Laziness of (:) leads to the conflation of lists and co-lists, making e.g., length a partial function.
• The conflation of streams with (co-)lists is likely historical and for convenience (and maybe to write beautiful papers about clever value-recursive programs). But it makes head, tail, !! etc. partial which are total functions on streams.

I think a systematic fix of the problem should introduce a Stream type into base where head, tail etc. can be total. Streams would be the target of functions like iterate and repeat.
Just taking away head and tail would be very inconvenient for those that mean streams when they say [a], because head :: Stream a -> Maybe a is just utter nonsense (albeit in a different way than head :: List a -> a).

[konsumlamm]

While I like the idea, I'm not sure it makes sense to create a Data.Maybe.Partial module just for a single function, Data,Maybe already is very small.

[soupi]

Another thing that should be taken into account in this proposal - many learning resources introduce head and tail.

[googleson78]

I strongly agree with what @andreasabel said - we should have, long-term, separate types for at least a finite list and streams. I think that that effort is however much larger and that this proposal might even be a step in that direction and/or part of the process (once we've cleaved the functions that are valid on Streams into a separate place it will be much easier to push people to use the total versions that operate on Streams), and additionally, this proposal by itself has great value, so I would like to see this happen first.

[Gabriella439]

I don't believe that adding other list-like types to Prelude should be a precondition for removing the partial versions from base

My reasoning is that if the Prelude did not already have these partial functions, I'm fairly confident the committee would not accept adding such partial functions if they were proposed today

[Ericson2314]

@andreasabel Yes I have wanted Stream at least since using https://hackage.haskell.org/package/clash-prelude-1.6.3/docs/Clash-Signal.html, but I agree with @Gabriella439 that this shouldn't be blocked on having Stream exist. Besides the reason she said, here are a few more:

1. Even if we had Stream we would still need Data.List.Partial to have the smooth migration. I don't think an auto-migration tool could deal with the changing types, let alone decide whether one actually wanted Stream or NonEmpty since they both e.g. make head and tail safe.

2. The design here is fully compatible with adding Stream later. (Right?)

3. I want to establish precedent that migration tools alone can "unstuck" us with old controversies. We have many other situations where "nice situation to new learners" and "less breakage for old time users to fix" are in conflict, and I would like every time to use auto-migration tools to avoid a tradeoff and have our cake and eat it too.

That last one, while more of a meta concern, is most important to me. I think in the past we've done a bad job at understanding extent to which are conflicting priorities, and thus haven't been able to muster the energy to do things which would actually defuse the conflict (better migration tools!). I am really hoping we can turn a new leaf.

[andreasabel]

2. The design here is fully compatible with adding Stream later. (Right?)

Indeed so!

[goldfirere]

A subtlety in @andreasabel's point is that what counts as partial depends on your viewpoint. If you primarily think of [a] as a stream, then head is total and find is partial (even though it returns a Maybe). So I think we're really saying here: label functions as partial if they can fail on finite data. Or (perhaps equivalently?), label functions as partial if they can throw an exception, but pretend that potentially non-terminating functions are total. To be clear, I'm in support of this viewpoint -- I don't want to have to go through a Partial module to get length! -- but it is an opinion that we're establishing.

I'd also like to add a small spin to the requirements for the migration tool: that it can be called as a source plugin. That is, I want to be able to write -fplugin=Migration.ImportPartials (or whatever) on the command line, in a cabal.project or in an OPTIONS_GHC to perform a transformation to my source that will allow it to compile with e.g. import Data.List (head) in perpetuity. This is particularly important to authors, who can now post the incantation on their errata page, instead of providing detailed instructions for how to repair all of the examples in their book.

Happily, this requirement should not be hard to accommodate, because any migration tool essentially has to do all of this work already. I'm just specifying a particular interface. Note also that GHC already has support for code transformations that are intended to preserve comments, indentation structure, etc., so phrasing the migration tool as a plugin might even simplify the task.

[Ericson2314]

So I think we're really saying here: label functions as partial if they can fail on finite data.

Well to be really pendantic, it's fail in finite steps.

data X = Foo X | Bar X
foos (Foo x) = () : foos x

bad = let x = Bar x; in foos x

here foos has no finite data to fail on, but it is failing in finite steps.

I bring this up basically to say that Haskell never tries to save you from non-termination, but should try to say you from what we were calling "incompleteness" in new incomplete (missing patterns in case, missing fields in constructor, etc.)

The good thing about this is that it is an objective principle, not mattering on ones interpretation of what is [a] is for: Even if [a] is idiomatically infinite, trying to get the user to write correct co-terminating functions is out of scope, however failing to handle [] (the empty list) where it allowed to occur is always dubious style.

[Ericson2314]

-fplugin=Migration.ImportPartials

My preferred stretch goal would instead be shim libraries: if the user includes a build-depends base < 4.19, and we can make a base 4.19.x that depends on the newer base, the longevity story is even better! No errata is needed: cabal just downloads the shim and everything works.

Still this reminded me it's good to think in terms of what is blocking each step. I split step 1 into 1 and 2 to better leverage this:

1. The new home can be done immediately.
2. The deprecation can be done once we have deprecated reexports.
3. The final removal can happen after whatever other migration tools we want at also ready.

This means, sure, plugin and shim base why not! What I am really concerned with is getting to steps 1 and 2. If 3 doesn't happen for years and years so be it. At the same time, I think getting to step 2 will make this process "feel real", and the psychological boost of finally dealing with this longstanding bugbear will inspire us to write whatever migration tools we deem are necessary.

[goldfirere]

I agree with what you just wrote @Ericson2314 -- in particular, about having step 3 be far in the future.

So maybe the current proposal should just have steps 1 and 2, keeping step 3 aspirational. That is, we commit to

1. Making a new home for these functions
2. Deprecating exports once that is possible

but that's all we commit to. In general terms, the proposal can also say that we would like to actually remove the deprecated bits in the future, but that depends on a migration tool, or plugin, or reinstallable base, or some other mitigation strategy. Because we don't know yet what mitigations will actually be possible (that is, actually implemented -- they're all possible), we don't commit to any one. Furthermore, it would take a new proposal to hash out and agree to the specifics.

I have to admit that removing a concrete plan for (3) makes me considerably more enthusiastic about this proposal. I'm scared of (3), precisely because we don't exactly know what (3) looks like yet.

[Ericson2314]

@goldfirere Sounds great to me! Increasing enthusian at the mere cost of making (3) a bit more nebulous is a phenomenal trade as far as I am concerned :).