OK. But in concret terms ?
These points are also important and cans be translated at architecture / UX / team / ecosystem levels.
But let’s keep it simple with code.
Pure, total functions
Don’t lie to your users, allow them to react efficiently :
Explicit error channel
It’s a signal, make it unambiguous, give agency, automate it
Which intent is less ambiguous ?
Use the type system to automate classification of errors ?
“A type system is a tractable syntactic method for proving the absence of certain program behaviors by classifying phrases according to the kinds of values they compute.” — Benjamin Pierce
By definition, a type system automatically categorize results
⟹ need for a dedicated error chanel + a common error trait
Same for effectful functions !
Use a dedicated error channel :
~ Either[E, A] for pure code,
else ~ IO[E, A] monad
Model everything ?
→ Where is the limit ?
Need for a systematic approach to error management
A school of systems
BOUNDED group of things, with a NAME, Interacting with other systems
Systems have horizon
Nothing exists beyond horizon
Systems have horizon. Horrors lie beyond.
Horizon limit is your choice — by definition
⟹ SecurityException is an expected error case here
… but nowhere else in Rudder. By our choice.
A bit more about systems
Need for a systematic approach to error management
BOUNDED group of things with a NAME Interacting via INTERFACES by a PROTOCOL with other systems and PROMISING to have a behavior.
Make promises, keep them
*make errors relevant to their users
*there is NO definitive answer
*discuss, share, iterate
Unpopular opinion : Checked exceptions are good signal for users. Who likes them ?
What’s missing for good error management in code ?
*exception are a pile of ambiguity
*no tooling, no inference, nothing
→ you need to be able to manipulate errors like normal code
→ where are our higher order functions like map, fold, etc ?
→ loose referential transparency*
Make it joy !
*automatic (in for loop + inference)
*or as expressive as nominal case!
*composition (referential transparency…)
*higher level resource management: bracket, etc
*add all the combinators you need!
*it’s cheap with pure, total functions
You still have to think in systems by yourself, then ZIO provides :
→ create: from Option, Either, value…
→ transform: mapError, fold, foldM, ..
→ recovery: total, partial, or else
→ .bracket / Managed, asyncqueues, STM, etc : safe, composable resource management
Everything work in parallel, concurrent code too !
Inference just work !
One Error hierarchy
Generic useful errors
Specialized error for subsystems
Full example :
→ That’s a very good point and you should ALWAYS try to do so. The idea is to change method’s domain definition (ie, the parameter’s shape) to only work on inputs that can’t rise errors. Typically, in my trivial “divide” example, we should have use “non zero integer” for denominator input.
→ Alexis King (@lexy_lambda) wrote a wonderful article on that, so just go read it, she explains it better than I can
“Parse, don’t validate” https://lexilambda.github.io/blog/2019/11/05/parse-don-t-validate/
→ We use that technique a lot in Rudder to drive understanding of what is possible. Each time we can restrict domain definition, we try to keep that information for latter use.
→ Typical example: parsing plugin license (we have 4 “xxxLicenses” classes depending what we now about its state); Validating user policy (again several “SomethingPolicyDraft” with different hypothesis needed to build the “Something”).
→ the general goal is the same than with error management: assess failure mode, give agency to users to react efficiently.
→ There’s still plenty of cases where that technique is hard to use (fluzzy business cases…) or not what you are looking for (you just want to tell users that something is the nominal case, or not, and give them agency to react accordingly).
→ no, SystemError is here to translate Error that need to be dealts with (like connection error to DB, FS related problem, etc) but are encoded in Java with an Exception. SystemError is not used to catch Java “OutOfMemoryError”. These exception kills Rudder. We use the JVM Thread.setDefaultUncaughtExceptionHandler to try to give more information to dev/ops and clean things before killing the app.
→ This is a very pertinent question, and we spend a log of time pondering between the current design and one where all sub-systems would have their own error type (with no common super type). In the end, we settled on the current design because:
1 — no common super type means no automatic inference. You need to guide it with transformer, and even if ZIO provide tooling to map errors, that means a lot of useless boilerplate that pollute the readability of your code.
2 — there is common tooling that you really want to have in all errors (Chained, SystemError, but also “notOptional”, etc). You don’t want to rewrite them. Yes type class could be a solution, but you still have to write them, for no clear gain here.
3 — you are fighting the automatic categorization done by the compiler in place of leveraging it.
4 — The gain (detailed error) is actually almost never needed. When we switched to “only one super class for all error”, we saw that “Chained” is sufficient to deals with general trans-system cases, and in some very, very rare cases, you case build ad-hoc combinators when needed, it’s cheap.
→ So all in all, the wins in convenience and joy of just having evering working without boilerplate clearly outpaced the not clear gain of having different error hierarchies.
→ The problem would have been different if Rudder was not one monolithic app with a need of separated compilation between services. I think we would have made an “error” lib in that case.
→ Well, the decision to switch is yours, and I don’t know the specific context of your company to give an advice on that. Nonetheless, here is my personal opinion:
1 — ZIO stack seems simpler (less concepts) and work perfectly with inference. Thus it may be simpler to teach it to new people, and to maintain. YMMV.
2 — ZIO perf are excellent, especially regarding concurrent code. Fibers are a very nice abstraction to work with.
3 — ZIO enforce pure code, which is generally simpler to compose/refactor.
4 — ZIO tooling and linked construction (Managed resources, Async Queues, STM, etc) are a joy to code with. It removes a lot of pains in tedious, boring, complicated tasks (closing resources correctly, sync between concurrent access, etc)
5 — pertinent stack trace in concurrent code is a major win
But at the end of the day, you decide!
→ It’s complicated :). 1 month of part time (me), plus lots more time for teaching, refactoring, understanding new paradigm limits, etc
1 — we didn’t started from nowhere. We were using Box from liftweb, and a lot of the code in Rudder was already “shaped” to deal with errors as explain in the talk (see https://issues.rudder.io/issues/14870 for context)
2 — we didn’t ported all Rudder to ZIO. I estimated that we ported ~ 40% of the code (60k-70k lines ?).
3 — we did some major refactoring along the lines, using new combinators and higher level structures (like async queues)
4 — we started in end of 2018, when ZIO code was still moving a lot and we switch to new things we when became available (ZIO 1.0.0 is around the corner and it as been quite stable for months now)
5 — we spent quite some time looking for the best choice for errors between sub-system (see other question)
Pour lire la 2nd partie du meetup, par François Laroche (Lead Developer Scala chez Make.org), c’est par ici !
Date de publication : 23 mars 2020