As one who frequently deals with Scala code, I have to say that this guy is right: the Scala community in particular seems to relish in complexity.
Scala is a wonderful language and well-designed, especially in contrast to Java. The learning curve requires a good deal of effort, but with experience, it becomes incredibly fun to write Scala code. Once I got past the learning curve, Scala became my go-to language.
That said, there’s an underlying arrogance of Scala developers who, in my experience, tend to often abuse the language to show off their cleverness at the expense of users and maintainers. I don’t know why this is. I suspect it’s because Odersky set the bar so high with his obvious brilliance that some feel insecure.
One take away quote for me:
“Unfortunately, we don’t like boring. We like to take the language and the compiler to their limits. We also like to adopt the latest fad. But the software systems we work on in our jobs should not be a playground or a test bed for the latest fad.”
Scala developers aren’t doing anything new here. People have abused fads in programming for decades. “Design Patterns” and “Test Driven Development” and “RESTful” come to mind. However, it’s unfortunate that Scala developers in particular do this. Because, Scala seems to be a language specifically designed to avoid past mistakes.
I think the equivalent to this in Python is "How many list comprehensions can I nest!?"
Solving things with list comprehensions is very idiomatic python, so the thought process goes "the more of these I can stuff inside each other, the more idiomatic my code is."
And `explicit is better than implicit` is idiomatic python, but the language uses the most implicit characters available on the average keyboard (` ` and ` `) to denote control flow.
In that sense control flow only exists as the relation between other things, so it makes sense for it to be denoted that way. If a line with no code on it is indented in the forest and nobody hears it, does the line execute?
I write my python interactively in a repl, and often end up with deeply nested list comprehensions as a result. I always need to break them up to have any chance of understanding it even a day later.
I weep anytime I need to decode a one-liner list comprehension/map/filter/reduce/lambda abomination. Like... would it have killed you to break it into multiple lines and sprinkled in some comments?
I worked professionally for a year with Scala. For me the biggest problem were functional fanatics that design solutions only top 20% of devs can understand. The rest comes from the object oriented realm and can't deal with monad transformers etc. That might work in small teams, but when you want to scale a team to 100 people you can't ignore this effect. And functional fanatics are strong in ignorance.
I worked for years in Scala, and the biggest problem I had was OOP fanatics who clung to stale design patterns and out of date knowledge of general programming principles despite FP being both the simpler, cleaner, more legible, and more maintainable approach. Part of this is the fault of the individual not asking more of themselves, and part of this is the language, encouraging people to feel good about their FP+OOP hybridity, which is entirely incompatible as a concept. I've never had a new grad who could not make the switch to an FP mindset. I have, however, had older or more established devs who don't want to learn and thought they could get away with shit work (which is the main cause of scalability issues, not new concepts).
For the record, I never used more complicated concepts than Monads and MTL in my day to day usage. If you can deal with Monads, you can deal with MTL. If you can't deal with Monads, then you didn't put in the 1 whole day of work it takes to become comfortable with the concept. If you can learn what an AbstractFactoryBean is, you can damn sure learn that a Monad consists of 4 functions across 3 interfaces and that Functor:fmap => Applicative:(pure + ap + fmap) => Monad(>>= + pure + ap + fmap).
> functional fanatics that design solutions only top 20% of devs can understand
Not a part of that group, but I think this is a false narrative. 20% might be accurate, or it might be a bit high even. But it has nothing to do with `top` or `bottom` performers. It's purely to do with exposure to functional styles.
Learning Haskell first doesn't make you a top 20% developer, much as I'm sure HN in particular will want to argue with me on that point.
All I'm trying to say is: there's not an intelligence spectrum of `imperative` to `functional` with `object oriented` in the middle. They're different, and many who learn one first struggle with the others, but that doesn't mean they're lower performers in what they do know than the ones who can talk about currying vs partial application.
I mean, those are much closer to functional than imperative tbh. At best it's simply too abstracted to discuss either way. Recipes would be a better example.
If you want to use that example it'd be more like this snippet that no one on earth would tell their children:
> Starting from n = 0, take the nth vegetable from your plate, insert your fork into that vegetable, raise your fork to your mouth, bite that vegetable, remove your fork from your mouth, masticate, add one to n, and repeat.
You can get really far on Scala just by following a couple of principles.
1) Aim towards your functions depending only on the function inputs, to calculate a return value, with no side effects
2) Aim towards your functions having a return type that exactly describes the possible outputs of your inputs.
A lot of the magic after that is clever composition that, while having the benefit of making your code shorter and smaller and more "expressive", is also obfuscating to people that are less experienced.
This is another way of saying "it's too complicated".
The purpose of the advancement of technology is to make things easier. The practical use of these languages is to create something that makes money, with as little overhead as possible. A language should be selected to make things easier, with little overhead. Not being able to easily scale to 100 people is a very bad business decision, especially when the same functionality could be achieved in languages/methods that require less education (cheaper) and have a larger talent pool (cheaper). I'm talking about the use of tools that are meant for productivity, in general, not Scala.
I agree with you 100% plus, I'd like to take it up a notch by saying that Pure FP evangelists are the sole reason why Scala is not widely used.
Also, it is not seldom that you hear them say "It took me n years to fully grok the _whatever FP concept_". Or Martin Odersky doesn't get the _whatever FP concept_.
So how do they expect the majority of sw engineers will use/adopt those principles if the guy who invented Scala "does not get it" and even themselves took so much time to "get it"...???
I'm not sure that the right thing is that they relish in complexity:
They tend to use it as something to show off their ego. It's a bit too academic and it often fails to understand the user's need.
It's often ignored about how the tool/library integrates with their user's code. For example, Http4s wants you to form your application around it, rather than just being a single component of your application.
- You don't like "those people" (the Scala community)
- You explain behaviors of Scala programmers in terms of your beliefs of their internal states ("they relish complexity") rather than external factors.
“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?”
--Brian Kernighan
How many times have I bitten myself this way? Dear past me: stop being so smart.
I really like it when people become concerned about cognitive complexity. I think it's one of the significantly ignored concepts in our industry and if we ever find a way to address the issue (either with some sort of cognitive complexity solution or some sort of big oh like math framework that describes how things are complex) then the future generations of software engineers will look at what we're currently doing now the same way we look at how people used to do alchemy for our lack of being able to handle cognitive complexity.
Personally, I want a mathematical structure of "problems" such that we can track when we're actually making things more complex or less complex. In the article, removing boilerplate was a problem because internal domain models were being exposed publicly. But what I really want to know is "what are we actually gaining by removing the boiler plate and what are we losing by exposing domain models". In like a general mathematical sense such that if everyone about programming changes (maybe suddenly we switch to APL or something), the lessons we learned are still valid.
So far the best I've managed is [1]. I've done a bunch of refinement in the last year and a half, but the general structure is about the same. I'm not suggesting that what I have is complete or even necessarily the right direction to go for this sort of thing. But what I am saying is that in order to make definite statements about how the problem and/or code you're dealing with is complex (that don't rely on appeals to authority) then you probably need to have done at least as much "work" as I have (if that makes any sense).
I like the alchemy analogy. I think a huge trap we tend to fall into as a profession is presuming to know what software engineering practices are good or bad when we actually have next to zero empirical data on the matter. Most of what the accepted dogma is comes from either authority, or from what techniques lend themselves to the strongest arguments.
>> Hence, we try to isolate business logic from effects and implement the former as pure functions.
This is a valuable insight. Especially if your main programming language supports both functional and procedural and/or OOP approaches;
This statement offers a good guideline as to where the boundary should be.
I find that functional programming is great for business logic but not as great for more concrete logic where performance is important.
For example, when dealing with low level concrete objects like sockets.
It's a refreshing read even for non FP/Scala development world.
It's not hard to see that modern software engineering is cherishing accidental complexity, and as it comes from the "tools we use", the fair share of problem is in the languages, frameworks and tools we use. Unfortunately, there are very little of them that really take a tough stance on fighting complexity, instead of adding more features. Actually I can think only of one such language – Go, and, unsurprisingly, it's critizised heavily in many circles exactly for the lack of accidental complexity.
Tony Hoare wrote this in 1974: “But the pursuit of simplicity is one of the most difficult and challenging activities of the human mind. Progress is likely to be extremely slow, where each complexity eliminated must be hailed as a breakthrough. We need not only brilliance of intellect but breadth of experience, nicety of judgement, excellence of taste, and even more than our fair share of good luck.“
I was quite impressed after reading this essay (it's from "The software design: a parable" [1]) – written almost half a century ago, and there seem to be a little progress, if not negative. So books and talks are unlikely to change anything – we, people will still use the path of least resistance and always will be subjected to congitive biases – so probably only tooling can make a change. Go, for example, simply doesn't allow you to do many overly complex things (on purpose) and it results in extremely readable codebase – chances that you open random piece of Go code on internet and will not be able to understand it are almost zero.
I find it a terrific achievement, and wish other (future) languages could embrace the importance of it.
Scala is an especially messed up situation. Everything is either overengineered or simplistic; there is no good middle ground because the language is a mess.
This is a little-acknowledged industry-wide problem that extends beyond hot-shot overengineering to reems of banal code too, so I would rather it be written from a less expected perspective.
> Doing that certainly means that you scrap quite a bit of boilerplate and reduce the complexity in your code. However, if you look at the bigger picture, this is the opposite of removing complexity. Exposing your domain model like this almost always leads to strongly coupled systems that are very difficult to evolve. If you ask me, this is one of the most promising ways to create a distributed monolith. It really amazes me how many people are totally okay with simply making their internal model their public API, especially since functional programmers are usually crazy about abstractions.
Good functional programmers are happy not to introduce an extra layer of indirection yet, because they know that their code is easy to refactor. The article itself seems to come to that point later, yet it gets it completely backwards here.
Generate JSON formats the easy, zero-boilerplate way for now. As and when you have a requirement for a JSON representation that looks different from your domain representation - perhaps because you want to evolve your internal domain model while keeping the JSON representation the same - you can introduce manual JSON codecs (or DTOs, or whatever you prefer) at that time.
> Object-oriented programmers call this technique stubbing, and they get mocked for doing it, mostly by people like us.
They get mocked for doing it because it breaks all the rules of the language. It usually relies on reflection, bytecode manipulation, or other such horrors. It's often thread-unsafe. Having tests that you can't refactor the way you normally refactor code is a huge, mostly-hidden cost.
> But you have no clue if your production interpreter, which is talking to an external system, is working correctly at all. So abstracting over the effect monad and using the Id monad in tests doesn’t even help you to write meaningful tests.
Tests need to be decomposed like any other code. There's a huge value in testing your business logic separately from testing your interactions with external systems. Functional core, imperative shell achieves that, but at a higher cost: you can't interleave your interactions with external systems and your logic in the natural way in code, you have to explicitly, manually push all the interactions to the edge instead.
> What if we did server-side rendering of HTML instead? There wouldn’t be any need to design an evolvable JSON API, which, unlike simply exposing your domain model, is not trivial, and there would definitely be no need for defining any JSON codecs. And that’s just what you get on the server side. On the frontend side, you would have one piece of software less to maintain, you’d get faster load times and you could get rid of duplicated business logic.
You'd get slower load times as you had to do page transitions, and more duplicate logic as things like form validation had to be done on the frontend and backend with separate code.
> At some point, we realize that we have an architectural problem at the macro level. Maybe, if we continue to ask why long enough, we will come to the conclusion that all these services we need to communicate with exist in exactly this way because of how the company is organized.
And which is easier to fix? At some point you decide that the entirety of western civilization is fundamentally flawed, and you can either start ranting about how we need to rebuild society from the ground up, or you can put a bit of complexity in your software and make something actually useful.
> I was recently talking to someone about Rust, and that person said, “A language without higher-kinded types is completely useless!”. Yes, certain abstractions are not possible without them, but I have never seen a project fail due to a lack of higher-kinded types.
That may have been me, and I absolutely have seen projects fail due to lack of higher-kinded types. Not in an obvious way; rather it felt like a general sludginess to development, a need to repeat the same tasks over again, until the project eventually dies from a thousand cuts. (Hell, for all I know the company I left several years ago may still be trying to make that project work).
> Projects fail because the problem domain is not well understood, or because the architecture is not a good fit for the requirements. They also fail because of how the organisation is structured and because of politics, be it intra- or inter-organizational. Projects fail because of people, and because of people not talking to each other.
This is facile. Projects fail because of people talking to each other too much. Projects fail because people tried to fix their organisational politics instead of implementing technical workarounds. Projects fail because the technology choices were simply bad.
Reading this article was weird, because I found myself nodding along with most of the woolly principles but then massively disagreeing with every single example. It makes me think the article is arguing at the wrong level of abstraction somehow: no-one thinks you should use a complex solution when a simple one will do. Very few people will claim that the problem with today's software is that programmers think things through too much and should rush to write code instead (though for the record I think that's actually the case). But those consensus principles don't actually help us at all when it comes to discuss specific decisions.
> Very few people will claim that the problem with today's software is that programmers think things through too much and should rush to write code instead (though for the record I think that's actually the case).
People coming from easy to refactor languages have widely different rule books from the ones coming from boileplate-filled ones. It expected that the same person thinks people should rush to write code earlier and gets to understand how a project can fail because of lack of higher-kinded types. The hard part is explaining to people how those weird languages make one more productive, because their thought follows completely opposite rules.
Yeah, I really wanted to agree with the author and I did on some points e.g. avoid to abstract unnecessarily over effects. But advocating for explicitly defined library interfaces is actually dangerous unless there is the specific need for a stable interface, as it adds complexity in the form of technical debt.
I was pleasantly surprised to read an article like this. Less surprised that it came from Daniel Westheide. He know's his stuff.
What I was expecting: I was expecting the fad pushing an article about how too many people push components together and then they're trying to sell some kind of microservices nonsense.
As one who frequently deals with Scala code, I have to say that this guy is right: the Scala community in particular seems to relish in complexity.
Scala is a wonderful language and well-designed, especially in contrast to Java. The learning curve requires a good deal of effort, but with experience, it becomes incredibly fun to write Scala code. Once I got past the learning curve, Scala became my go-to language.
That said, there’s an underlying arrogance of Scala developers who, in my experience, tend to often abuse the language to show off their cleverness at the expense of users and maintainers. I don’t know why this is. I suspect it’s because Odersky set the bar so high with his obvious brilliance that some feel insecure.
One take away quote for me:
“Unfortunately, we don’t like boring. We like to take the language and the compiler to their limits. We also like to adopt the latest fad. But the software systems we work on in our jobs should not be a playground or a test bed for the latest fad.”
Scala developers aren’t doing anything new here. People have abused fads in programming for decades. “Design Patterns” and “Test Driven Development” and “RESTful” come to mind. However, it’s unfortunate that Scala developers in particular do this. Because, Scala seems to be a language specifically designed to avoid past mistakes.
I think the equivalent to this in Python is "How many list comprehensions can I nest!?"
Solving things with list comprehensions is very idiomatic python, so the thought process goes "the more of these I can stuff inside each other, the more idiomatic my code is."
Solving things with list comprehensions is very idiomatic in Python.
Also, not nesting things is very idiomatic Python ("Flat is better than nested" is part of the Zen).
So, people generally don't think of nested list comprehensions as idiomatic.
And `explicit is better than implicit` is idiomatic python, but the language uses the most implicit characters available on the average keyboard (` ` and ` `) to denote control flow.
The language uses indentation levels to denote control flow, and those are very explicit - much more so than braces in fact.
They're not though. They only exist as the relation between the edge and the start of content.
How many levels of indentation are there on each line?
https://i.imgur.com/s1WBHxU.png
Spoiler: Line 3 has no indentation while Line 6 has one level. Line 4 has a mix of tabs and spaces that would confuse Python.
The fact of the matter is that indentation is white space, and white space is implicit by definition. It only exists as the gap between other things.
Stop using a legacy version of Python.
You will get a "TabError: inconsistent use of tabs and spaces in indentation" exception if you use Python 3.
I haven't used python with inconsistent indentation recently, that's true.
In that sense control flow only exists as the relation between other things, so it makes sense for it to be denoted that way. If a line with no code on it is indented in the forest and nobody hears it, does the line execute?
I write my python interactively in a repl, and often end up with deeply nested list comprehensions as a result. I always need to break them up to have any chance of understanding it even a day later.
I weep anytime I need to decode a one-liner list comprehension/map/filter/reduce/lambda abomination. Like... would it have killed you to break it into multiple lines and sprinkled in some comments?
I worked professionally for a year with Scala. For me the biggest problem were functional fanatics that design solutions only top 20% of devs can understand. The rest comes from the object oriented realm and can't deal with monad transformers etc. That might work in small teams, but when you want to scale a team to 100 people you can't ignore this effect. And functional fanatics are strong in ignorance.
I'll do you one better.
I worked for years in Scala, and the biggest problem I had was OOP fanatics who clung to stale design patterns and out of date knowledge of general programming principles despite FP being both the simpler, cleaner, more legible, and more maintainable approach. Part of this is the fault of the individual not asking more of themselves, and part of this is the language, encouraging people to feel good about their FP+OOP hybridity, which is entirely incompatible as a concept. I've never had a new grad who could not make the switch to an FP mindset. I have, however, had older or more established devs who don't want to learn and thought they could get away with shit work (which is the main cause of scalability issues, not new concepts).
For the record, I never used more complicated concepts than Monads and MTL in my day to day usage. If you can deal with Monads, you can deal with MTL. If you can't deal with Monads, then you didn't put in the 1 whole day of work it takes to become comfortable with the concept. If you can learn what an AbstractFactoryBean is, you can damn sure learn that a Monad consists of 4 functions across 3 interfaces and that Functor:fmap => Applicative:(pure + ap + fmap) => Monad(>>= + pure + ap + fmap).
> functional fanatics that design solutions only top 20% of devs can understand
Not a part of that group, but I think this is a false narrative. 20% might be accurate, or it might be a bit high even. But it has nothing to do with `top` or `bottom` performers. It's purely to do with exposure to functional styles.
Learning Haskell first doesn't make you a top 20% developer, much as I'm sure HN in particular will want to argue with me on that point.
All I'm trying to say is: there's not an intelligence spectrum of `imperative` to `functional` with `object oriented` in the middle. They're different, and many who learn one first struggle with the others, but that doesn't mean they're lower performers in what they do know than the ones who can talk about currying vs partial application.
>many who learn one first
Everyone learns imperative first. "Wake up!" or "Eat your vegetables!" are imperative. No-one says to their kids, "map(vegetables, eat)"
I mean, those are much closer to functional than imperative tbh. At best it's simply too abstracted to discuss either way. Recipes would be a better example.
If you want to use that example it'd be more like this snippet that no one on earth would tell their children:
> Starting from n = 0, take the nth vegetable from your plate, insert your fork into that vegetable, raise your fork to your mouth, bite that vegetable, remove your fork from your mouth, masticate, add one to n, and repeat.
You can get really far on Scala just by following a couple of principles.
1) Aim towards your functions depending only on the function inputs, to calculate a return value, with no side effects
2) Aim towards your functions having a return type that exactly describes the possible outputs of your inputs.
A lot of the magic after that is clever composition that, while having the benefit of making your code shorter and smaller and more "expressive", is also obfuscating to people that are less experienced.
If you are trying to write Java in Scala, you probably shouldn't be using Scala.
The problem is mostly a documentation problem and education problem.
This is another way of saying "it's too complicated".
The purpose of the advancement of technology is to make things easier. The practical use of these languages is to create something that makes money, with as little overhead as possible. A language should be selected to make things easier, with little overhead. Not being able to easily scale to 100 people is a very bad business decision, especially when the same functionality could be achieved in languages/methods that require less education (cheaper) and have a larger talent pool (cheaper). I'm talking about the use of tools that are meant for productivity, in general, not Scala.
It's a more precise way of saying it's too complicated. You should be precise if you want to solve issues.
> methods that require less education (cheaper)
This is not a good thing. Trying to devalue a skill is just asking for trouble, especially in engineering.
Don't functional fanatics write code that has no side effects?
That's the goal.
I agree with you 100% plus, I'd like to take it up a notch by saying that Pure FP evangelists are the sole reason why Scala is not widely used.
Also, it is not seldom that you hear them say "It took me n years to fully grok the _whatever FP concept_". Or Martin Odersky doesn't get the _whatever FP concept_.
So how do they expect the majority of sw engineers will use/adopt those principles if the guy who invented Scala "does not get it" and even themselves took so much time to "get it"...???
I'm not sure that the right thing is that they relish in complexity:
They tend to use it as something to show off their ego. It's a bit too academic and it often fails to understand the user's need.
It's often ignored about how the tool/library integrates with their user's code. For example, Http4s wants you to form your application around it, rather than just being a single component of your application.
Your argument, distilled:
- You don't like "those people" (the Scala community)
- You explain behaviors of Scala programmers in terms of your beliefs of their internal states ("they relish complexity") rather than external factors.
- "Those people" are arrogant.
- "Those people" are abusive.
- "Those people" show off
- "Those people" exploit users and maintainers
- "Those people" are insecure
- "Those people" aren't doing anything new.
- "Those people" are just following fads.
You're right, they're really doing it for job security...
“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?”
--Brian Kernighan
How many times have I bitten myself this way? Dear past me: stop being so smart.
This is an astounding quotation
I really like it when people become concerned about cognitive complexity. I think it's one of the significantly ignored concepts in our industry and if we ever find a way to address the issue (either with some sort of cognitive complexity solution or some sort of big oh like math framework that describes how things are complex) then the future generations of software engineers will look at what we're currently doing now the same way we look at how people used to do alchemy for our lack of being able to handle cognitive complexity.
Personally, I want a mathematical structure of "problems" such that we can track when we're actually making things more complex or less complex. In the article, removing boilerplate was a problem because internal domain models were being exposed publicly. But what I really want to know is "what are we actually gaining by removing the boiler plate and what are we losing by exposing domain models". In like a general mathematical sense such that if everyone about programming changes (maybe suddenly we switch to APL or something), the lessons we learned are still valid.
So far the best I've managed is [1]. I've done a bunch of refinement in the last year and a half, but the general structure is about the same. I'm not suggesting that what I have is complete or even necessarily the right direction to go for this sort of thing. But what I am saying is that in order to make definite statements about how the problem and/or code you're dealing with is complex (that don't rely on appeals to authority) then you probably need to have done at least as much "work" as I have (if that makes any sense).
[1] - https://www.sep.com/sep-blog/2017/04/25/objective-code-quali...
I like the alchemy analogy. I think a huge trap we tend to fall into as a profession is presuming to know what software engineering practices are good or bad when we actually have next to zero empirical data on the matter. Most of what the accepted dogma is comes from either authority, or from what techniques lend themselves to the strongest arguments.
Has somewhat similar intro to the Rich Hickey's famous "Simple made easy" talk. A good read nevertheless.
https://www.infoq.com/presentations/Simple-Made-Easy
My favorite part is the knitted castle.
...and that's IMO a good thing.
Well, it feels borderline weird -- since it's so obviously similar, it could use a reference.
>> Hence, we try to isolate business logic from effects and implement the former as pure functions.
This is a valuable insight. Especially if your main programming language supports both functional and procedural and/or OOP approaches; This statement offers a good guideline as to where the boundary should be.
I find that functional programming is great for business logic but not as great for more concrete logic where performance is important. For example, when dealing with low level concrete objects like sockets.
It's a refreshing read even for non FP/Scala development world.
It's not hard to see that modern software engineering is cherishing accidental complexity, and as it comes from the "tools we use", the fair share of problem is in the languages, frameworks and tools we use. Unfortunately, there are very little of them that really take a tough stance on fighting complexity, instead of adding more features. Actually I can think only of one such language – Go, and, unsurprisingly, it's critizised heavily in many circles exactly for the lack of accidental complexity.
Tony Hoare wrote this in 1974: “But the pursuit of simplicity is one of the most difficult and challenging activities of the human mind. Progress is likely to be extremely slow, where each complexity eliminated must be hailed as a breakthrough. We need not only brilliance of intellect but breadth of experience, nicety of judgement, excellence of taste, and even more than our fair share of good luck.“
I was quite impressed after reading this essay (it's from "The software design: a parable" [1]) – written almost half a century ago, and there seem to be a little progress, if not negative. So books and talks are unlikely to change anything – we, people will still use the path of least resistance and always will be subjected to congitive biases – so probably only tooling can make a change. Go, for example, simply doesn't allow you to do many overly complex things (on purpose) and it results in extremely readable codebase – chances that you open random piece of Go code on internet and will not be able to understand it are almost zero.
I find it a terrific achievement, and wish other (future) languages could embrace the importance of it.
[1] https://people.dsv.su.se/~jpalme/s1/hoare.pdf
Scala is an especially messed up situation. Everything is either overengineered or simplistic; there is no good middle ground because the language is a mess.
This is a little-acknowledged industry-wide problem that extends beyond hot-shot overengineering to reems of banal code too, so I would rather it be written from a less expected perspective.
> Doing that certainly means that you scrap quite a bit of boilerplate and reduce the complexity in your code. However, if you look at the bigger picture, this is the opposite of removing complexity. Exposing your domain model like this almost always leads to strongly coupled systems that are very difficult to evolve. If you ask me, this is one of the most promising ways to create a distributed monolith. It really amazes me how many people are totally okay with simply making their internal model their public API, especially since functional programmers are usually crazy about abstractions.
Good functional programmers are happy not to introduce an extra layer of indirection yet, because they know that their code is easy to refactor. The article itself seems to come to that point later, yet it gets it completely backwards here.
Generate JSON formats the easy, zero-boilerplate way for now. As and when you have a requirement for a JSON representation that looks different from your domain representation - perhaps because you want to evolve your internal domain model while keeping the JSON representation the same - you can introduce manual JSON codecs (or DTOs, or whatever you prefer) at that time.
> Object-oriented programmers call this technique stubbing, and they get mocked for doing it, mostly by people like us.
They get mocked for doing it because it breaks all the rules of the language. It usually relies on reflection, bytecode manipulation, or other such horrors. It's often thread-unsafe. Having tests that you can't refactor the way you normally refactor code is a huge, mostly-hidden cost.
> But you have no clue if your production interpreter, which is talking to an external system, is working correctly at all. So abstracting over the effect monad and using the Id monad in tests doesn’t even help you to write meaningful tests.
Tests need to be decomposed like any other code. There's a huge value in testing your business logic separately from testing your interactions with external systems. Functional core, imperative shell achieves that, but at a higher cost: you can't interleave your interactions with external systems and your logic in the natural way in code, you have to explicitly, manually push all the interactions to the edge instead.
> What if we did server-side rendering of HTML instead? There wouldn’t be any need to design an evolvable JSON API, which, unlike simply exposing your domain model, is not trivial, and there would definitely be no need for defining any JSON codecs. And that’s just what you get on the server side. On the frontend side, you would have one piece of software less to maintain, you’d get faster load times and you could get rid of duplicated business logic.
You'd get slower load times as you had to do page transitions, and more duplicate logic as things like form validation had to be done on the frontend and backend with separate code.
> At some point, we realize that we have an architectural problem at the macro level. Maybe, if we continue to ask why long enough, we will come to the conclusion that all these services we need to communicate with exist in exactly this way because of how the company is organized.
And which is easier to fix? At some point you decide that the entirety of western civilization is fundamentally flawed, and you can either start ranting about how we need to rebuild society from the ground up, or you can put a bit of complexity in your software and make something actually useful.
> I was recently talking to someone about Rust, and that person said, “A language without higher-kinded types is completely useless!”. Yes, certain abstractions are not possible without them, but I have never seen a project fail due to a lack of higher-kinded types.
That may have been me, and I absolutely have seen projects fail due to lack of higher-kinded types. Not in an obvious way; rather it felt like a general sludginess to development, a need to repeat the same tasks over again, until the project eventually dies from a thousand cuts. (Hell, for all I know the company I left several years ago may still be trying to make that project work).
> Projects fail because the problem domain is not well understood, or because the architecture is not a good fit for the requirements. They also fail because of how the organisation is structured and because of politics, be it intra- or inter-organizational. Projects fail because of people, and because of people not talking to each other.
This is facile. Projects fail because of people talking to each other too much. Projects fail because people tried to fix their organisational politics instead of implementing technical workarounds. Projects fail because the technology choices were simply bad.
Reading this article was weird, because I found myself nodding along with most of the woolly principles but then massively disagreeing with every single example. It makes me think the article is arguing at the wrong level of abstraction somehow: no-one thinks you should use a complex solution when a simple one will do. Very few people will claim that the problem with today's software is that programmers think things through too much and should rush to write code instead (though for the record I think that's actually the case). But those consensus principles don't actually help us at all when it comes to discuss specific decisions.
> Very few people will claim that the problem with today's software is that programmers think things through too much and should rush to write code instead (though for the record I think that's actually the case).
People coming from easy to refactor languages have widely different rule books from the ones coming from boileplate-filled ones. It expected that the same person thinks people should rush to write code earlier and gets to understand how a project can fail because of lack of higher-kinded types. The hard part is explaining to people how those weird languages make one more productive, because their thought follows completely opposite rules.
Yeah, I really wanted to agree with the author and I did on some points e.g. avoid to abstract unnecessarily over effects. But advocating for explicitly defined library interfaces is actually dangerous unless there is the specific need for a stable interface, as it adds complexity in the form of technical debt.
I was pleasantly surprised to read an article like this. Less surprised that it came from Daniel Westheide. He know's his stuff.
What I was expecting: I was expecting the fad pushing an article about how too many people push components together and then they're trying to sell some kind of microservices nonsense.
See also: http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLe...
Scala is for people who weren’t around for C++ template metaprogramming.
tl;dr https://en.m.wikipedia.org/wiki/Chunking_(psychology)