When I was in Japan it was strange to see how slow some trains are. You will see that sometimes they stay more time in a station and will accelerate way faster than usual.
But when one train arrives it is common that there is another one arriving at the same time. People exchange lines from one train into another in a few seconds and then both of them depart.
My feeling is that the optimization is not for the speed of the trains. Japanese trains do not go as fast as they can. But all the train system is optimized to reduce waiting time. Each individual train is slightly slower. But the mean time that Japanese travellers spend is reduced.
I would love to see some study about this, as it was just my travel experience. It would be cool to compare that on how ants behave. It will not be the first time that ants have resolved an optimization problem way before even humans existed.
I noticed something similar in Switzerland. First time we went there, it was before cell phones, and we went from A to B by going to the station and telling them where we wanted to go. We were given tickets and a little itinerary. At one station, our "layover" was 1 minute. I protested: "We will never catch our next train." The agent looked at us and said: "You will catch your train." Sure enough, we did.
The other thing I noticed was that when one of our trains was delayed a bit, it seemed like they slowed down the entire system so that nobody would miss a connection, and then made it back up on the trip to the next station. We never missed a connection, and the only times when we had to hustle, was because we were nervous Americans accustomed to US air travel. Even the buses were synchronized with the trains.
In Eliyahu M. Goldratt's "Theory of Constraints", you can't increase the throughput of a factory by making each step in the manufacturing process faster [1,2].
That only results in building up inventory which actually decreases the efficiency of the entire process.
Instead, identify and fix the biggest bottleneck. Even if you do that inefficiently, with more labor, or with a sub-optimal machine, fixing the biggest bottleneck increases total throughput.
Once that's done, then focus on the biggest bottleneck of the newly revised process.
The counterintuitive result is that by focusing only on constraints, the process steps that are unconstrained will be idle much of the time.
Inexperienced factory managers see people standing around doing nothing and think this is inefficiency to be eliminated. Experienced managers realize that idleness is essential at unconstrained stations, and trying to eliminate it will actually decrease total output.
This reminds me of the story where a Houston airport kept getting complaints from airline passengers that it took too long for their luggage to arrive. So they moved baggage claim farther away. This made travelers walk farther to get to their bags, but have a shorter "wait" time. Complaints dropped to zero.
I wonder if there is a cultural difference there, where throughput is understood to be better than latency or something.
This explains why Japanese train conductors make a big deal out of delays of a few seconds. It’s very common to hear announcements like “sorry we were delayed by 20 seconds because someone was holding the door.” Because that delays the entire line and all connecting trains by 20 seconds. And I’m sure these delays are tracked to measure performance of the conductors.
> If this distribution of labor operates this way in your office, however, (the 30 percent may laugh knowingly now), there’s no real solace to be taken from the ant experiment, unless you are digging a tunnel perhaps. Assuming everyone has their own computer, phone and cubicle, they could all be working all the time.
I think this generalizes to knowledge work better than the author gives credit for. You don't have literal traffic jams (well, maybe you do with commuting) but you can have too many people in one meeting. Or too many coders touching the same file. Or too many designers on a product. And so on.
I was thinking the same thing and came to post something similar: haven't we already come to the conclusion that throwing more bodies at a knowledge problem does not allow the problem to be solved any faster ?
Scheduling theory says you need idle units to take up required work, otherwise your latency increases by the time remaining for the next task completion.
Given how poorly people multitask, I'm surprised things like devops aren't over-filled so that failures that need responding to don't affect schedules.
I'm reading through "The Phoenix Project", and this is one of the items mentioned, is that the less idle time a resource has, the higher the work backlog (WIP). Of course, in a factory setting (as presented in the book), I can see this working -- although you have an expensive machine, you aren't paying it by the hour to while it is idle. Of course, for devops work, if you don't want your human resources to be idle, then I could see scheduling easily-interruptible tasks (such as reading tech articles or HN) during idle periods, so that when a task is ready for you to work on you can jump right to it.
This is what we're doing. We're distinguishing project work, which requires longer and more constant focus on a problem and other, smaller stuff. For example, setting up configuration management for an entirely new application is a project. Writing and deploying an icinga check, a telegraf collector and such is smaller stuff.
With this, we schedule some people to be on project duty and they shall not be interrupted. Other guys are on interrupt duty and tend to do tasks which just require an hour to half a day of attention so they can easily close something they're doing and jump on something urgent.
> Given how poorly people multitask, I'm surprised things like devops aren't over-filled so that failures that need responding to don't affect schedules.
In my experience, most management is unfamiliar with things like scheduling theory. That they'd fail to overfill some areas to cover peak demand is not surprising to me.
An idle worker is an evil thing to most management minds, they don't understand that 100% activated workers means you have no capacity to handle surplus work. And every "stop the world" event to cover a new fire (started by management's bad judgement) means that the fires will continue.
The title is misleading. As far as the article goes, this pertains only to tunnel digging. That's at least in part because the "dig, move rubble, dig" task is vulnerable to congestion.
Congestion is an issue if you're using the painter's algorithm to dig the tunnel (pick up a chunk of dirt from the end of the tunnel, carry it all the way back to the beginning, trudge back down to the end, pick up a second chunk of dirt...). Any number of ants can apply that strategy in parallel, but they'll be stymied by congestion in a tight tunnel.
But that algorithm is terrible. Most of your time is wasted walking back and forth. You can do a lot better by setting up a relay, where one ant is responsible for a fixed length of tunnel, and its job is to pick up dirt from one end and drop that dirt at the other end for the next ant.
Congestion is still a problem for that strategy, but the whole issue is different in kind. This task scales better with more ants than the stupid way -- the more ants you have, the shorter you can make each relay leg, without having two ants contend for the same physical space at any point.
> Most of your time is wasted walking back and forth
If the tunnel is two ants wide then the relay is basically the same as a long circuit. Every bit of dirt needs to be carried the entire length of the tunnel, so it doesn't really matter if one and carries it the whole way or if n ants do -- the number of ant-miles full and empty are the same, it's just the relative cost of passing manoeuvres and handoffs that matter (or nonlinear carrying costs -- maybe it's easier on the ants to have shorter, more frequent changes between carrying and not carrying.)
What if the tunnel is two ants wide and seven relay-legs long? You can usefully employ 14 ants that way, where you'd be stuck at two without the relays.
You're right about the ant-miles full and empty being constant; I guess the advantage of the relay is purely the ability to scale without congesting.
If anyone still uses centralized version control where only one person can check out a given file at a time, that's the epitome of programmer congestion.
This reminds me of how you are supposed to keep a sizable percentage of an investment portfolio in cash, earning nothing, so it is available when the time comes to make further investments in a market downturn.
How do the ants coordinate the 70/30 ratio? I'm assuming there isn't much communication going on, rather that there's some procedure they follow so that the ratio occurs emergently.
Reminds me of an article on crows, and how some that were slackers most of the time would step in to take care of the young when the more active members of the group got ill.
So the ant-secret to alleviating vehicular traffic is to just have 70% of drivers just stay at home.
Anyone who's work in a large corporation has probably noticed about 30% of the employees do 70% of the work.
I often read comments here saying they don't understand why more IT folk don't work from home. Turns out we may have been approaching the problem from the wrong angle.
We ought be espousing the benefits of not-working from home.
Well, sort of. The ant-secret is to simply create more space and fewer opportunities for collisions. Humans, the supposedly "superior" creatures to ants, are terrible at driving because most of them treat driving as a competition, ignore established flow of traffic (either the speed limit or the current average speed), and switch lanes too often, among other things; all of it might only save a minuscule amount of time for one driver while incidentally slowing down all the other cars on the road.
Traffic would almost be a much lesser for humans if they drove at a reasonable speed and kept a greater distance between cars to compensate for the poor human reaction time. Of course, as you noted, keeping cars off the road would be the most realistic way of achieving this as there's no way to convince the average person to drive properly.
I don't think that most tech workers should have to work in an office, but it would make more sense for them to work in an office than to employ the 70-80% of ineffective workers and force them to commute every day. But then those millions of people would have no jobs or means of income!
Perhaps, then, there could be a good argument made for universal basic income; calculate all the money lost in traffic collisions, the resulting deaths, maintenance of roads, fossil fuels, future expenses caused by global warming, etc., and perhaps it would cost us more than to pay most of the population not to work or at least not work in offices.
I get a lot more work done from home than I do in my distraction filled open office hellscape.
I also don’t get hit by errant nerf darts from the new batch of SDEs that act like the office is a daycare.
There’s nothing valuable about the modern corporate tech office space. Any work accomplished there is in spite of the work environment, not because of it.
It's a "startup" thing and since corporate America loves to sell the idea that we are all startups it happens in a lot of places. And it's stupid. Just like open offices. This article makes it about gender equality but I'm male and I still don't think it is fun.
"Daniel I. Goldman, a physicist at the Georgia Institute of Technology, and his colleagues, found that the secret to efficient tunnel digging by fire ants was that 30 percent of the ants did 70 percent of the work."
When I was in Japan it was strange to see how slow some trains are. You will see that sometimes they stay more time in a station and will accelerate way faster than usual.
But when one train arrives it is common that there is another one arriving at the same time. People exchange lines from one train into another in a few seconds and then both of them depart.
My feeling is that the optimization is not for the speed of the trains. Japanese trains do not go as fast as they can. But all the train system is optimized to reduce waiting time. Each individual train is slightly slower. But the mean time that Japanese travellers spend is reduced.
I would love to see some study about this, as it was just my travel experience. It would be cool to compare that on how ants behave. It will not be the first time that ants have resolved an optimization problem way before even humans existed.
I noticed something similar in Switzerland. First time we went there, it was before cell phones, and we went from A to B by going to the station and telling them where we wanted to go. We were given tickets and a little itinerary. At one station, our "layover" was 1 minute. I protested: "We will never catch our next train." The agent looked at us and said: "You will catch your train." Sure enough, we did.
The other thing I noticed was that when one of our trains was delayed a bit, it seemed like they slowed down the entire system so that nobody would miss a connection, and then made it back up on the trip to the next station. We never missed a connection, and the only times when we had to hustle, was because we were nervous Americans accustomed to US air travel. Even the buses were synchronized with the trains.
In Eliyahu M. Goldratt's "Theory of Constraints", you can't increase the throughput of a factory by making each step in the manufacturing process faster [1,2].
That only results in building up inventory which actually decreases the efficiency of the entire process.
Instead, identify and fix the biggest bottleneck. Even if you do that inefficiently, with more labor, or with a sub-optimal machine, fixing the biggest bottleneck increases total throughput.
Once that's done, then focus on the biggest bottleneck of the newly revised process.
The counterintuitive result is that by focusing only on constraints, the process steps that are unconstrained will be idle much of the time.
Inexperienced factory managers see people standing around doing nothing and think this is inefficiency to be eliminated. Experienced managers realize that idleness is essential at unconstrained stations, and trying to eliminate it will actually decrease total output.
[1] https://en.wikipedia.org/wiki/Theory_of_constraints
[2] https://en.wikipedia.org/wiki/The_Goal_(novel)
So basically: they optimize for throughput, not latency.
This reminds me of the story where a Houston airport kept getting complaints from airline passengers that it took too long for their luggage to arrive. So they moved baggage claim farther away. This made travelers walk farther to get to their bags, but have a shorter "wait" time. Complaints dropped to zero.
I wonder if there is a cultural difference there, where throughput is understood to be better than latency or something.
https://www.nytimes.com/2012/08/19/opinion/sunday/why-waitin...
This explains why Japanese train conductors make a big deal out of delays of a few seconds. It’s very common to hear announcements like “sorry we were delayed by 20 seconds because someone was holding the door.” Because that delays the entire line and all connecting trains by 20 seconds. And I’m sure these delays are tracked to measure performance of the conductors.
> If this distribution of labor operates this way in your office, however, (the 30 percent may laugh knowingly now), there’s no real solace to be taken from the ant experiment, unless you are digging a tunnel perhaps. Assuming everyone has their own computer, phone and cubicle, they could all be working all the time.
I think this generalizes to knowledge work better than the author gives credit for. You don't have literal traffic jams (well, maybe you do with commuting) but you can have too many people in one meeting. Or too many coders touching the same file. Or too many designers on a product. And so on.
The Mythical Man-Month :)
I was thinking the same thing and came to post something similar: haven't we already come to the conclusion that throwing more bodies at a knowledge problem does not allow the problem to be solved any faster ?
Scheduling theory says you need idle units to take up required work, otherwise your latency increases by the time remaining for the next task completion.
Given how poorly people multitask, I'm surprised things like devops aren't over-filled so that failures that need responding to don't affect schedules.
If you could hatch IT staff from eggs and keep them half-starved in a coma until needed, they probably would be.
Dont give my boss any ideas.
I'm reading through "The Phoenix Project", and this is one of the items mentioned, is that the less idle time a resource has, the higher the work backlog (WIP). Of course, in a factory setting (as presented in the book), I can see this working -- although you have an expensive machine, you aren't paying it by the hour to while it is idle. Of course, for devops work, if you don't want your human resources to be idle, then I could see scheduling easily-interruptible tasks (such as reading tech articles or HN) during idle periods, so that when a task is ready for you to work on you can jump right to it.
This is what we're doing. We're distinguishing project work, which requires longer and more constant focus on a problem and other, smaller stuff. For example, setting up configuration management for an entirely new application is a project. Writing and deploying an icinga check, a telegraf collector and such is smaller stuff.
With this, we schedule some people to be on project duty and they shall not be interrupted. Other guys are on interrupt duty and tend to do tasks which just require an hour to half a day of attention so they can easily close something they're doing and jump on something urgent.
Or, seen from the other way, the higher the work backlog, the less idle time a resource has...
I suspect ants don't have accountants who view things like devops as a cost centre.
If the availability of tasks and failures that need responses was a source of nutrients that caused things like devops staff to reproduce...
It's true: all analogies break down under close inspection.
Not all. 70% of those ants are gonna go broke and get their homes foreclosed if they don’t get off their ass and get a job.
DevOps is the managers dream where the engineer does operations on top of his engineering workload. Less people less cost.
> Given how poorly people multitask, I'm surprised things like devops aren't over-filled so that failures that need responding to don't affect schedules.
In my experience, most management is unfamiliar with things like scheduling theory. That they'd fail to overfill some areas to cover peak demand is not surprising to me.
An idle worker is an evil thing to most management minds, they don't understand that 100% activated workers means you have no capacity to handle surplus work. And every "stop the world" event to cover a new fire (started by management's bad judgement) means that the fires will continue.
Try recruiting 20 devops people, can't do that overnight, especially outside of tech centers.
A single anthill has more idle ants than there will ever be devops/sysadmins in the world.
Surely this depends on the anthill — the common species near me only reach a few thousand individuals per colony...
The title is misleading. As far as the article goes, this pertains only to tunnel digging. That's at least in part because the "dig, move rubble, dig" task is vulnerable to congestion.
Congestion is an issue if you're using the painter's algorithm to dig the tunnel (pick up a chunk of dirt from the end of the tunnel, carry it all the way back to the beginning, trudge back down to the end, pick up a second chunk of dirt...). Any number of ants can apply that strategy in parallel, but they'll be stymied by congestion in a tight tunnel.
But that algorithm is terrible. Most of your time is wasted walking back and forth. You can do a lot better by setting up a relay, where one ant is responsible for a fixed length of tunnel, and its job is to pick up dirt from one end and drop that dirt at the other end for the next ant.
Congestion is still a problem for that strategy, but the whole issue is different in kind. This task scales better with more ants than the stupid way -- the more ants you have, the shorter you can make each relay leg, without having two ants contend for the same physical space at any point.
> Most of your time is wasted walking back and forth
If the tunnel is two ants wide then the relay is basically the same as a long circuit. Every bit of dirt needs to be carried the entire length of the tunnel, so it doesn't really matter if one and carries it the whole way or if n ants do -- the number of ant-miles full and empty are the same, it's just the relative cost of passing manoeuvres and handoffs that matter (or nonlinear carrying costs -- maybe it's easier on the ants to have shorter, more frequent changes between carrying and not carrying.)
What if the tunnel is two ants wide and seven relay-legs long? You can usefully employ 14 ants that way, where you'd be stuck at two without the relays.
You're right about the ant-miles full and empty being constant; I guess the advantage of the relay is purely the ability to scale without congesting.
> you'd be stuck at two without the relays.
You mean if you had two ants walking side-by-side? If it's two ants wide you can have one lane in each direction and use all the ants you like.
Is programming vulnerable to congestion?
If anyone still uses centralized version control where only one person can check out a given file at a time, that's the epitome of programmer congestion.
Merges
Sure, check out 'garbage collection'.
reminds me of a recent article about how Messi walks more than other players https://fivethirtyeight.com/features/messi-walks-better-than...
This reminds me of how you are supposed to keep a sizable percentage of an investment portfolio in cash, earning nothing, so it is available when the time comes to make further investments in a market downturn.
That's opposite of what I've heard.
The phrase I've always heard is "Time in the market beats timing the market".
Of course, it depends on who you listen to.
How do the ants coordinate the 70/30 ratio? I'm assuming there isn't much communication going on, rather that there's some procedure they follow so that the ratio occurs emergently.
ants do communicate with chemical trails of some sort.
Reminds me of an article on crows, and how some that were slackers most of the time would step in to take care of the young when the more active members of the group got ill.
The ants in the video appear to have been painted bright colors for enhanced visibility. It's quite striking.
I feel like most DoT road crews have this already figured out.
So the ant-secret to alleviating vehicular traffic is to just have 70% of drivers just stay at home.
Anyone who's work in a large corporation has probably noticed about 30% of the employees do 70% of the work.
I often read comments here saying they don't understand why more IT folk don't work from home. Turns out we may have been approaching the problem from the wrong angle.
We ought be espousing the benefits of not-working from home.
Well, sort of. The ant-secret is to simply create more space and fewer opportunities for collisions. Humans, the supposedly "superior" creatures to ants, are terrible at driving because most of them treat driving as a competition, ignore established flow of traffic (either the speed limit or the current average speed), and switch lanes too often, among other things; all of it might only save a minuscule amount of time for one driver while incidentally slowing down all the other cars on the road.
Traffic would almost be a much lesser for humans if they drove at a reasonable speed and kept a greater distance between cars to compensate for the poor human reaction time. Of course, as you noted, keeping cars off the road would be the most realistic way of achieving this as there's no way to convince the average person to drive properly.
I don't think that most tech workers should have to work in an office, but it would make more sense for them to work in an office than to employ the 70-80% of ineffective workers and force them to commute every day. But then those millions of people would have no jobs or means of income!
Perhaps, then, there could be a good argument made for universal basic income; calculate all the money lost in traffic collisions, the resulting deaths, maintenance of roads, fossil fuels, future expenses caused by global warming, etc., and perhaps it would cost us more than to pay most of the population not to work or at least not work in offices.
I get a lot more work done from home than I do in my distraction filled open office hellscape.
I also don’t get hit by errant nerf darts from the new batch of SDEs that act like the office is a daycare.
There’s nothing valuable about the modern corporate tech office space. Any work accomplished there is in spite of the work environment, not because of it.
>>> I also don’t get hit by errant nerf darts from the new batch of SDEs that act like the office is a daycare.
This is clearly an issue with your current employers that should not be generalized to all companies.
It's a "startup" thing and since corporate America loves to sell the idea that we are all startups it happens in a lot of places. And it's stupid. Just like open offices. This article makes it about gender equality but I'm male and I still don't think it is fun.
https://motherboard.vice.com/en_us/article/ez3mde/dear-start...
"Daniel I. Goldman, a physicist at the Georgia Institute of Technology, and his colleagues, found that the secret to efficient tunnel digging by fire ants was that 30 percent of the ants did 70 percent of the work."
That's astonishing. Why isn't it 20/80?
Or 1/99?
/s