> Tooling is so important to the developer experience for any language, and a language server is one of the most important kinds of tooling there is (because the user interacts with it on virtually every keystroke in their editor).
Here’s how languages backed by large corporations win. They have the resources and the optics to know why these non-fun features are important.
The irony is that the popularization of the language server protocol made it easier for small languages to get good tooling. Where before you had to either write your own IDE, or pick an existing one (of which all had upsides and downsides, so better pick wisely) and write an extension for it - often requiring learning a completely new language and framework to do so - now you can write a straightforward out-of-proc server implementation in the language itself (and likely reusing the existing parser etc that you already have) against a standard protocol, and have several popular IDEs and editors light up immediately.
To some extent; but there's solid reasoning behind this. By coming up with LSP, Microsoft achieved two important goals.
First, and most importantly, its languages (esp. TypeScript) can be used by people who already have an established workflow around an existing editor or IDE. It's much easier to sell a new language that plays well with an existing ecosystem, than it is to sell a whole new ecosystem (and Microsoft knows that full well from experience with .NET!). And higher adoption of TypeScript across the industry means that more people eventually come through it to the larger Microsoft ecosystem, as well, even if it's a small percentage of the larger crowd.
At the same time, it's also a way to recruit the open source community to extend Microsoft IDEs. If you're making a new PL or templating framework or something else that needs editing support, and your audience is primarily people on Vim and Sublime, then LSP is an obvious choice to minimize code duplication. But once you have it, you might as well package it for the VSCode extension gallery as well (and if you won't, someone else probably will).
So I think it's a win-win for everyone, except for companies that sell integrated all-in-one tooling solutions where the main benefit to the user is derived from only one part of it (because users can now mix and match as they see fit, and bundling becomes harder).
You can generalize this by saying that as production values go up and user expectations rise, it takes more work and a larger organization with more people to succeed.
This is true of programming languages, but just as true of movies, games, mobile apps, and so on. It's the result of competition in a creative endeavor where it's possible to scale by adding more people.
One way to avoid this is to come up with a different "indy" aesthetic where traditional production values aren't so important. Another way is to figure out how to build on the work of others instead of starting from scratch.
I don't see how it stands to reason that this is a function of large corporations. Rust has been further along this path given their relative ages and Mozilla is no where near the size or scope of Google. Likewise with a number of other languages that have official or otherwise LSs. Same story with Cargo out of the gate, vs Go modules after nearly 5 years of pain without them, etc.
Edit: I sound more negative than I meant. I am excited about this. It was cool that Sourcegraph and the VS Code extension pioneered here, but it's exciting to have official support and a place for everyone to rally for improvements. I look forward to seeing this used in things like Theia, VS Code, etc.
The reason Go took so long with modules is another function of large corporations. Google doesn’t use modules internally (they have the monorepo) so having a module system wasn’t important.
Rust may have been further along on this path, but in the grand scheme of things Go 1.1 feels far more “complete” than Rust 2015 - and a large part of that is Go’s huge standard library that Google could fund to develop.
There's a thread on rust internals where planning is taking place for a long-term strategy for RLS. It looks like end of 2019 might be a realistic time frame for a compiler-backed RLS. The good news is that it looks like this work might have some significant benefits for incremental compile times too :)
I agree Cargo is good but there are some issues that bug me (e.g. no support for docker layers with cargo --dependencies-only, xargo is not merged in, is there a support for custom repositories yet?).
Microsoft has always been very supportive of the developers in all the technologies they put on the market (Remember the "Developers!" from Ballmer?). They have the expertise in tools and company long-term requirements. Now that they are combining all that with a more open approach, we get Visual Studio Code, the Language Server Protocol, the buying of GitHub, etc. Interesting times for the developers!
This is ironic, in that tooling is exactly what large mainframe systems had early on. With extensive tooling can come lack of agility, which can be its own death sentence.
That said, an advantage of being backed by a large corporation is mind share. Having corporate spending on effectively propaganda helps. A lot.
>With extensive tooling can come lack of agility, which can be its own death sentence.
What do you mean by this? Without tooling you can get stuck in the muck doing everything by hand. Tools enable you to do more with less, so I don't really understand what you are getting at here.
Exactly. Hell, I consider tooling to be some of the most fun (and often difficult) things I work on in my free time.
If I can make something easier for my normal dev environment I see huge gains ~40 hours a week. That's pretty damn awesome and flashy to me. It's definitely not easy, but imo it's well worth my time.
It's used by editors like Visual Studio Code to provide intellisense, autocomplete, breakpoints and the like.
The idea is that instead of each editor writing it's own (incomplete) Python or Go plugin, someone writes a language server which is then used (and improved) by everybody.
To add to this: the beauty of the protocol is that it uses JSON over stdin/stdout for communication. So if your language can make console apps, you can write a language server in it that works with any compliant editor/IDE - no FFI necessary, no need to learn how to author extensions for that editor etc.
And, conversely, if your editor/IDE can fork a child process and redirect its standard streams, you can support any language server and not care about how it's implemented.
I think this more than anything else has contributed to its rapid adoption, especially on Unix-likes.
The protocol itself is transport-agnostic - it just needs a bidirectional stream, and the spec doesn't define what it is. In practice, both stdin/stdout and sockets are used - the latter usually for language servers that are "heavyweight", such that it is advantageous to spin up the server once, "prime" it (let it do any indexing etc), and then reuse it between concurrent sessions. The standard streams are usually the default choice for simplicity reasons, though.
Implementing a server in Brainfuck is theoretically possible, but only if you ignore the heat death of the universe as the upper time boundary. ~
mdempsky's fork of gocode should work with Go 1.11...
and rstambler's fork of that fork (https://github.com/stamblerre/gocode) should work with modules.
* edit: mdempksy's fork should work with Go 1.11 used outside of a module or with GO111MODULE=off; Rebecca's fork adds support for modules on top of that
I'm not sure I fully understand the provenance of the code here, but I could just be being a bit thick. The files in the change request show the copyright as "2018 The Go Authors", but some of the commentary hints that this is code donated from the Sourcegraph implementation. If this is the case, does that mean Sourcegraph has transferred the copyright to the Go authors?
I’m going to guess this is in response to many community supported dev tools breaking in v1.11 when modules are turned on. Specifically tools like gocode, godef, and gogetdoc.
Does anyone know if this is introducing more features than the existing Go LSP[1]? I'm all in favor of tooling support - I'm just curious if this actually adds anything to those of us already using a Go LSP. Thoughts?
If I'm reading the linked article right this is about moving exactly the go language server you are referring to into the Google maintained set of go core libraries and tools. So this first is independent from which features it supports. Now as it is open source, you bet both Google and sourcegraph will continue adding features as appropriate.
My understanding is that at least for the most popular languages, they were already doing more than the language server protocol would allow, so switching to language servers would be a downgrade.
That's their specialty, but I'm not sure it's all that relevant for users who prefer lightweight editors.
WebStorm definitely runs the TypeScript language service, but I believe they also have their own parser, completions, etc.
I think JetBrains is in a weird position. If they strictly rely on the official language server for everything, then they're not able to innovate or integrate with the many language-aware JetBrains features that go beyond what language servers typically do. On the other hand, the language server will always be a better source of truth for the basics since it's implemented by the language authors, and it requires no maintenance by JetBrains. My impression is they use a careful combination of both, but ideally it would be a unified implementation rather than two redundant implementations running side-by-side in different processes.
Why can they not extend the language server to meet their needs? Or if such feature is found useful generally for other editors too, then the language server should implement that than keep Jetbrains having some partial hacks on top of it.
The TypeScript language service is indeed extensible, but keep in mind that extensibility points can only get you so far, and JetBrains has spent the last 20 years putting a massive amount of work into a JVM-based code intelligence system. They can't just port all that to Node.js (and Go and every other language/runtime where language servers run).
Of course it would be great if the official TS language service included every imaginable IDE feature, but realistically it will always lag behind a team focused on exploring new ideas. JetBrains is going to want to experiment with new language-aware features, iterate on some of them, and throw out the ones that they aren't happy with. An open source tool shared by all editors isn't the right environment for that sort of experimentation. (They could fork the language service, though, which has its downsides but maybe is most reasonable here.)
the language server protocol doesnt support everything jetbrains does.
they can extend the server as much as they want, if the protocol doesn't support the additions, they'll be useless features unless they'll implement protocol extension, which could potentially be incompatible with the original... and they're back to the starting proposition.
I don't know if they "refuse" to leverage it. They have been building IDEs for a long time and have a toolchain for building parsers for an IDE. It would make sense for them to just use that. Besides, they were building the Go IDE before the LSP was mature.
very few languages implement the RLS well. For instance Rust does it, but its incomplete and very slow. I bought a quieter CPU fan specifically because of it!
It's more a function of language complexity than anything else. LSP for something like C, Java or TypeScript is not that hard, because the language is fairly straightforward to parse, and not ambiguous. But something like C++ or Rust, where you have to deal with templates/generics/traits/... and type inference starts getting gnarly. And something like JS or Python is hard because there are no static types at all, and you have to do inference.
But LSP itself doesn't really make it easier or harder. If you look at other code completion implementations (in IDEs that don't use LSP), they're usually at a comparable level, if they exist.
Good. Great even. If the implementation is top notch, mantained and from the 1st party, why the hell not, but most of the times, language server implementations are half assed, slow, "kinda" working messes that are all but useless in the long run. The PHP language server implementations are a great example of this. A buch of broken, incomplete, underperforming projects maintained by a single person (or straight up abandoned). Compare that with Phpstorm and you have just compared a rusty tricycle with a spaceship.
Because they're typically running locally they're quick, or at least as quick as your hardware will allow. The json-related latency is typically dwarfed by the latency from actually doing the functionality (parsing the code for code completion, etc).
So the real work is in having a good story for incremental compilation and a generally quick and robust compile toolchain. I guess that Go is in a good position here ...
The key word is "relatively". You're comparing it to protocols that are designed to process much heavier loads than what typing generates. And don't forget that processing is usually asynchronous, and can be spun off to a separate thread, so you get one just for protocol handling.
Good idea. Languages like Haskell benefit by having a language server that Emacs, VisualCode, etc. can all use. It just make good sense to do the work once and let all editors and IDEs use it.
The two major advantages of Go are its clear, unambiguous and explicit syntax (unfortunately ugly) and very small (minimal) feature set, so that it is possible to reason about anything without any high context.
The former means it is possible to pick up Go in a day or two and the latter means that there are no real tricks after a few months. That's it.
All this has been done in an excellent way. There is no class system. Everything is simple. Explicit typing works.
These design trade-offs have led to a language that is VERY easy to go back to after working in any other language. Almost all simple syntax mistakes by the programmer are caught before something successfully compiles. Once a programmer is used to it, which happens quickly, Go is the only language people write pages of code in that compiles on the first pass and does everything they hoped. "Less is exponentially more."
History
-------
Due to the history of Go it was meant to be a systems language. It didn't really succeed at replacing C anywhere, except the web server: I consider Go like a web-safe C. (In no small part due to Google's work meeting its own needs.)
Question 1
----------
What I don't understand is that since speed and being a systems language were explicit design goals, why the Go syntax is not suitable for tiny, embedded microcontrollers.
If the answer is something about garbage collection, this doesn't really make sense for me, as good algorithms should scale from 0 upward. I don't see why there should be a huge jump from "not running Go yet" to "now you're running a Go runtime" that suddenly requires a large amount of free memory. Here is a post about this:
I don't understand why Go doesn't take its core, primary strengths outlined in the "background" section above and expand it into the lowest of the low-end devices.
Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue? Go was designed as a compiled systems language, to replace C. Not as a scripting language to replace Python.
Question 2
----------
Since explicit (if very ugly) syntax that keeps people from making mistakes, and whose specifications can be kept in mind all at once, is a strength of the language and syntax, why can't I develop desktop GUI's in Go? It is a fast, compiled language.
You might say the answer to that is that a windowing system needs a class system but I disagree - that can't possibly be any more true than the idea that a web app needs a class system. Clearly people make web apps in Go without a class hierarchy.
Of course, when it debuted you could say, "That's just not what the team focused on", since it was backed by Google they needed it on the server. But at some point you'd expect first someone to make a make-do GUI toolkit, then other people to add on and make something better, and by now it should be an obvious language for making a Winzip or Winrar competitor. Or an antivirus. Or an Audacity or VLC-type program. Or an OpenOffice clone.
But nobody does these things. Any of these things.
Why not? The simple language has nothing missing that languages which are used for these applications have.
I don't buy the idea of the ecosystem entirely lacking these.
Why don't people make GUI stuff in Go? I don't understand why it shouldn't be an obvious language choice. I just wouldn't reach for Go when it comes to scripting a GUI application. Why not?
Summary / tl;dr
---------------
1. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for embedded microcontroller applications in 2018.
2. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for desktop applications in 2018.
I think these are good questions, even if they are off topic.
1. Why doesn’t Go seem to support embedded development. Off the top of my head there is a moderely successful embedded framework, gobot. The reason I don’t think it’s more popular is the runtime. Having any runtime at all tends to be poisonous and running Go without a runtime is difficult because you can’t manage your own memory in Go. Sure you could do a ton of work in making it work, but your effort is better spent just writing your project in Rust.
2. Why doesn’t Go support desktop? This is a case where I just don’t think anyways cares anymore about having good desktop support. You rarely see native desktop applications in anything other than C++ and C#. There are some UI toolkits out there but no one uses them, and something like Electron is used instead. Windows and OS X only give first class support to their blessed languages (C++, .NET, Objective C) and every other language you are out of luck.
I'm curious what you think the primary feature is that makes a Go runtime have to be necessarily large? (If writing a new Go runtime against the specification.)
You mention the need for a runtime, but I don't see that this makes it necessarily huge. For comparison, I'd like to think about using a TinyC compiler[1], called TCC, as a complete runtime to compare against since:
"
* TinyCC (aka TCC) is a small but hyper fast C compiler. Unlike other C compilers, it is meant to be self-relying: you do not need an external assembler or linker because TCC does that for you.
* TCC compiles so fast that even for big projects Makefiles may not be necessary.
* TCC not only supports ANSI C, but also most of the new ISO C99 standard and many GNUC extensions including inline assembly.
"
Now it turns out that the above fits in 100 KB and implements "ANSI C", which for the C89 version is 552 pages or so[2]. The Go specification is 73 pages or so in the same font. (So 13% of the size.) I also think that due to its design the Go language specification and syntax has fewer corner cases.
So clearly there is some "huge" feature that is really holding things back from running on a resource-constrained microcontroller in a stripped-down runtime. What is it?
To see where 100 KB fits (by analogy with TCC above), I looked up some microcontrollers. I first found a popular microcontroller for hobbyists, the ATmega2560 which runs the Arduino Mega. That microcontroller has 56KB ISP flash memory, 8KB SRAM. That RAM is not enough if we take the TinyC executable of around 100 KB as how much is needed. This microcontroller also only costs $6.5 in quantity 1 (unless I misread.)
Moving up a bit in price so we can get at least 128 KB of RAM, a popular choice is something like the AT32UC3A3256[3] which costs $31.25 in an evaluation kit (that gives you an additional 8 MB of RAM), and as far as shipping just the MCU (with its 128KB of RAM only), costs $12 in quantity one. That is a 32-bit device unlike the cheaper Atmega2560 I first mentioned (which is an 8-bit device) so should be easier for people to implement the standard against. The specifications are: 84 MHz, 256 kB program size, 128 kB RAM size.
Why can TCC be 100 KB executable size but independently implementing the Go specification can't?
As for why someone would want to, sure, you're going to be quite constrained, but if someone already knows Go, which had the design goal of being a general-purpose language, why can't they use its syntax for their microcontroller projects?
So what essential feature in the Go specification makes this an absolute non-starter? it's a super small standard.
(As for your second response, about "You rarely see native desktop applications in anything other than C++ ", I read what you wrote but I don't see why the Go ecosystem isn't slowly catching up anyway. It's a great language syntax.)
> So clearly there is some "huge" feature that is really holding things back from running on a resource-constrained microcontroller in a stripped-down runtime. What is it?
Garbage collection and goroutines.
You could probably strip back the implementations a bit but you can't get rid of gc - there is no explicit way of freeing memory. Possibly you could ban the go keyword and run on a single thread, although you'd have to deal with parts of the stdlib that use it.
Java and Oberon fit easily in a ARM Cortex M, which some embedded companies are happily selling, there is no reason why a Go implementation could not exist as well.
Do you think the specification is "correct" for a compiled language suitable for embedding?
The word "garbage" is in the specification a single time, in the introduction which says "garbage-collected". I can't see any constrained implementation details thereafter - see for yourself at where I might have missed it:
The gouroutine section (under the Go keyword) is three paragraphs: "A "go" statement starts the execution of a function call as an independent concurrent thread of control, or goroutine, within the same address space." and "The function value and parameters are evaluated as usual in the calling goroutine, but unlike with a regular call, program execution does not wait for the invoked function to complete. Instead, the function begins executing independently in a new goroutine. When the function terminates, its goroutine also terminates. If the function has any return values, they are discarded when the function completes. "
This does not seem to require the typical accoutrements of the Go compiler. If this is the complete verbiage specifying the behavior of a Goroutine, I could imagine a compliant implementation being lighter-weight than the official Go compiler.
Returning to the idea of GC: the word "memory" appears in the standard 4 times: once mentioning initialization of variables; once on the section on slices ("The built-in functions append and copy assist in common slice operations. For both functions, the result is independent of whether the memory referenced by the arguments overlaps. "); and twice in the last section on system considerations, including light MAY language in "Computer architectures may require memory addresses to be aligned; that is, for addresses of a variable to be a multiple of a factor, the variable's type's alignment. The function Alignof takes an expression denoting a variable of any type and returns the alignment of the (type of the) variable in bytes." and finally under Size and Alignment guarantees, which makes no reference to GC.
If this document is the complete specification, then it seems that the implementer has very wide leeway for how to implement the details of memory management via garbage collection. There doesn't seem to be much language limited behavior.
What do you think?
(In this comment I consider "Go" the language as anything that complies with the Go specification.)
To point 2) I fully agree, Go should be a great choice for desktop applications. One match I think is especially good, that is Gtk based development with Go. Gtk is one of the few UI toolkits which has been designed around a pure C API, so Go is a great match with it. Also, with Gtk the Go applications are reasonably portable to Linux, macOS and Window. I have used Go+Gtk and found it a pleseant experience.
You removed too much praise. Go's syntax is amazing. Can you put that part back in?
(Maybe you removed it because I also called it ugly - as opposed to a beautiful syntax like Python's - but that was not a design goal and doesn't bother me. Nobody meant for Go's syntax to be beautiful.) The summary you got down to "I appreciate Go since" is too weak.
Its syntax is the best of any language on planet Earth, which is why people can write pages of Go that work on first compilation and always do exactly what they imagine. (The second fact that leads to this is that its specifications and feature set is very small, so that it can be kept in mind by the programmer entirely.)
So add stronger praise. If not for such strong praise, why would I want to write everything from desktop applications to microcontroller code in Go, and not any other syntax?
I also have a very strong impression that its syntax and specifications are appropriate for both memory-sensitive applications and for desktop applications. (Not really a question). It is due to this strong impression that I want to know what the impediments are. (Maybe this was not clear.)
I can see no reason why a Go executable can't be made targeting 100 KB of RAM to do its GC and everything else. I also see no reason why bindings against QT or other GUI frameworks aren't made in the Go language syntax - this is a mystery to me. There must be some impediment.
What is it?
Can you reply with another draft that incorporates these?
I greatly appreciate the thoughts you added to the draft. I think Go is the best language syntax for general programming in any environment from a microcontroller with 150 KB of RAM to a major, 2 Gigabyte desktop behemoth's scripting language. I don't think the lack of class system is an issue (which is the only thing that comes to mind for me.)
So you need much stronger praise, please. Go is beautiful (despite its visually ugly syntax) and its use for web apps shows/proves its total appropriateness to all applications.
I want to understand the impediments. It should be much farther along to replacing C and C++.
Also:
>But I worry that Go is moving away from this mantra over the initial specification
I don't have such a worry. I guess you could say:
"But my impression is that the Go compiler leaves a lot of the huge benefit that is still in its specification: that is, a safer replacement for C and C++. It is perfectly viable as a replacement for nearly all C and C++ language work."
I'd like to understand why it isn't doing so yet and what it will take to get there. Hope this helps clarify my thinking.
Not going to edit the above but people clearly hate it (many downvotes without minutes of posting).
At the moment I think most Go programmers who also program tiny microcontrollers with a few hundred KB or up to a couple of MB of RAM, do so in languages like C or Rust, not in Go. Why not? It's a fair question.
For those of you reading this on a laptop or desktop, how many desktop applications are you all running at the moment or have as an installed application? How many are written in Go? Why so few or none?
I mean desktop applications like Audacity for audio, or VLC, or Open Office, or your web browser, and so forth.
The downvotes came pretty fast so you clearly have some ideas. I'd like to hear them.
Maybe because it's overlong, almost completely unrelated to the article, and might just be flamebait?
But I'll toss in a few cents anyway.
>Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue?
Microcontroller toolchains are pretty controller-specific. It takes work to make things compile for them. There are other languages that fare better. Additionally, one of the more useful features of Go (concurrent goroutines) would simply be unavailable (and even with concurrency disabled, you still get unpredictable latencies from deciding which task should run when, the GC, etc).
It's common to write apps using whatever the OS provides or using well-known libraries, so that's something Go's offerings don't offer. It's so much work already to build a GUI app that most people doing so probably don't want to concurrently write a GUI library too. I don't think there's anything stopping you, though.
Next time, avoid writing so much to ask two (three?) simple questions. You come off as having an attitude when you use phrases like "Not going to edit the above", "knights in shining armor", and so on. You also build up and tear down many bales worth of strawmen, which is pointless. It would have been a better use of your time to actually look into either of those fields and discover the answers for yourself.
Thanks for your responses, which I read carefully. I hope we can get a bit more out of this conversation.
"Just use C" is odd to me given that Go was originally developed to be an alternative to C (a "systems language") - this was an explicit design goal, and mentioned in the first main sentence of its specification[1]: "Go is a general-purpose language designed with systems programming in mind."
So when you then glance at the adoption for those uses, you would come to the (wrong) conclusion that it must have failed to meet its original design goal of being better for those general use cases. (Or people would be using it). According to its designers, C and C++ programmers don't transition to Go (as its designers had thought they would), but Python programmers do.
So I'd like to understand what it lacks that is holding that back. I see it as clearly superior. Looking at the design, it looks like it met its design goal. So I see it as successful in its bid to be design that is a viable replacement for C as a "systems language", and then despite this successful design now nobody uses it as a systems language.
It would be as though someone set out to make a better bicycle wheel, succeeded, and then nobody uses it as a bicycle wheel (despite its being a better bicycle wheel), while meanwhile the design is being used to haul thousands of tons of freight per year (server microservices). I'm asking, "so why don't we use it as bicycle wheels" and am told "just use a wheel". But I want to understand why, because it seems to me it met its design goal of being a better bicycle wheel.
For the second statement, your statement about "common to write apps using whatever the OS provides or using well-known libraries," makes sense and explains why these libraries didn't exist when Go was made. I don't understand why they are still not a big part of the ecosystem nearly a decade later.
The "knights in shining armor" is a serious statement: the biggest benefit of the Go language is its gatekeepers keeping things out of it, and its extremely orthogonal, tiny syntax, which can be kept in mind along with state. The results of code can be reasoned about without a lot of gotchas. I am therefore wondering what, if anything, these gatekeepers are keeping out which prevents the two parts of the ecosystem which I mentioned, from flourishing.
I don't think Go programmers who are also C and C++ programmers who are used to writing for embedded microcontrollers or for Windows in C and C++ respectively, require deep (or any) changes to the language before as a language it would be suitable to those two cases. You haven't mentioned any.
So I still welcome more viewpoints. I don't see why Go can't replace C and C++ applications across all of the use cases I mentioned, with minimal first-party support by the language developers and maintainers. Doesn't Go also have alternative Go compilers based on the specification?
As a followup, why is this the case? I mean, the Specification is quite clear, and it's a tiny language, so in the past 9 years I don't see why there aren't at least a few dozen independent implementations. Is it just not enough time, and we should wait a few decades? Is the issue that Google doesn't need Go for its desktop or mobile development stuff, and doens't need it for microcontrollers, and Go is not really an "independent" entity that is quite independent of Google as a company?
Basically I am wondering why Go in 2018 isn't where C and C++ were by 1998, for example. Despite your detailed answers I don't see the missing pieces.
Go as a syntax is better than C or C++ as a syntax. Its syntax and small and stable language features set are Go's primary advantages, which allows it run on the metal using clearly written standard Go libraries to serve web apps. To me, a microcontroller program running with next to no memory just has to not leak memory or crash, unlike Go running microservices, which have to withstand highly targeted and sophisticated attacks.
Basically, given how I see Go (which is the reason for my very lengthy introduction, so that people could correct me if I'm wrong) I don't see why the language is not very far along where C and C++ blossomed as languages for embedded microcontrollers, and for Desktop applications.
In a way you could say I don't really get what's preventing this. I can't think of any showstoppers.
I tried to find a succinct question to respond to for your first section, and couldn't. My answer is the same as before: no one has bothered to write a Go runtime that can run on microcontrollers. This is because Go does not do anything fundamentally different from C at that level, except things that are less suitable for low-memory and low-CPU environments; and because it takes a lot of work to write a compiler and runtime for these devices. Note that there are very few compilers for each individual microcontroller. They generally only have one or two.
>I see it as clearly superior.
You are not looking hard enough. Go has flaws, as does any language or tool. It is almost a rule of thumb that not understanding the flaws of a tool is a sign of inexperience with it or alternatives to it. I think if you actually tried to learn and use C and C++, you might better understand the difference.
>I don't see why Go can't replace C and C++ applications
My answer is the same as before: it can, except that C/C++ apps already have that territory, and Go doesn't provide a fundamentally different experience except in lacking libraries for most GUI applications. There are still places it shines, though. For example, it is doing a fine job of replacing C/C++ for cloud orchestration tools; static linking is important to those users, and Go provides it easily.
>Basically I am wondering why Go in 2018 isn't where C and C++ were by 1998, for example.
This is a mathematical error. The C programming language was developed in 1972, and C++ piggy-backed off of C's ecosystem very effectively.
>Go as a syntax is better than C or C++ as a syntax
I don't think I can come up with a better example of a "skin-deep" comparison, nor do I think anyone has actually argued for C/C++'s syntax being actually good.
>stable language features
Didn't Go 1.11 break all of the community's tools? Isn't that why the story we're posting on the comments on exists?
>To me, a microcontroller program running with next to no memory just has to not leak memory or crash,
Some microcontroller programs need to have very low latencies, or the physical things they are connected to will break (the GC defeats this). Some microcontroller programs are only allowed a couple of KB of RAM (Go can't handle that). Most microcontroller programs are compiled without a standard library (and Go would have to provide its own impls of basic Go stuff in order to function). Some microcontroller programs never allocate memory, cannot allocate memory, or must allocate all memory up front. And so on. You can solve these things by having more expensive and fault-tolerant gear, or buying more RAM, or paying people to write more Go compilers/runtimes, etc., but these things get expensive fast; that is often incompatible with the design goals of what microcontrollers are used for. You're welcome to write Go and slap it on a Raspberry Pi, though.
>In a way you could say I don't really get what's preventing this.
Try writing a microcontroller program and GUI app, both in C or C++ and in Go. I very strongly doubt you will succeed faster in Go than in C/C++, in either kind of project, because of multiple factors: ecosystem, communities, example code, language features, existing toolchains and compilers... But if you don't believe me, try it out for yourself.
...can't you possibly implement in only a couple of KB of RAM? (For what required feature or features can we say for sure that a couple of KB of RAM is insufficient to implement as specified?)
As it turns out, the only mention of the feature is in that link's introduction. The garbage collector (https://github.com/golang/go/blob/master/src/runtime/mgc.go) makes many references to 4KB pages and 1MB arenas. This (the code and the user of the technique) would need some serious reconsideration for small devices.
I don't downvote you but ... No one hate your post. It's just an huge wall of text unrelated to the current post. It seems not the good place or the good moment to discuss this. You can maybe try to do an "Ask HN" for open question like this.
I think it's a great place to ask this. The most fantastic people to answer the question are going to be reading a post like this, and what we're reading about (adding a Go language server to core) is a huge and language-specialized feature for core. I look forward to reading people's answers to my questions.
I thought the downvotes were because of something I got very wrong. Anyway I hope to learn from responses even if I got something wrong.
Didn't downvote your post above, so that wasn't me.
That aside, there are some resources around for this though I've not (yet) tried any of them:
* http://embd.kidoman.io - Go Embedded Programming Framework. Seems to support Raspberry Pi and BeagleBone Black.
* https://gobot.io - Robotics more than embedded, but there seems like some overlap. :)
---
With the "Why no desktop apps in Go?" question, for me it's because I'd only do so using something like Qt (cross platform).
While there have been several attempts to get usable Go bindings happening for Qt, none of them seem both really well done + long term stable.
The best developed Qt bindings so far are https://github.com/therecipe/qt. But that seems to have tapered off and the developer doesn't seem interested in making it into a team project (eg long term viability). That's my impression anyway, I could be wrong. :)
> Tooling is so important to the developer experience for any language, and a language server is one of the most important kinds of tooling there is (because the user interacts with it on virtually every keystroke in their editor).
Here’s how languages backed by large corporations win. They have the resources and the optics to know why these non-fun features are important.
The irony is that the popularization of the language server protocol made it easier for small languages to get good tooling. Where before you had to either write your own IDE, or pick an existing one (of which all had upsides and downsides, so better pick wisely) and write an extension for it - often requiring learning a completely new language and framework to do so - now you can write a straightforward out-of-proc server implementation in the language itself (and likely reusing the existing parser etc that you already have) against a standard protocol, and have several popular IDEs and editors light up immediately.
The ironic thing is that the LSP was started by a corporation.
To some extent; but there's solid reasoning behind this. By coming up with LSP, Microsoft achieved two important goals.
First, and most importantly, its languages (esp. TypeScript) can be used by people who already have an established workflow around an existing editor or IDE. It's much easier to sell a new language that plays well with an existing ecosystem, than it is to sell a whole new ecosystem (and Microsoft knows that full well from experience with .NET!). And higher adoption of TypeScript across the industry means that more people eventually come through it to the larger Microsoft ecosystem, as well, even if it's a small percentage of the larger crowd.
At the same time, it's also a way to recruit the open source community to extend Microsoft IDEs. If you're making a new PL or templating framework or something else that needs editing support, and your audience is primarily people on Vim and Sublime, then LSP is an obvious choice to minimize code duplication. But once you have it, you might as well package it for the VSCode extension gallery as well (and if you won't, someone else probably will).
So I think it's a win-win for everyone, except for companies that sell integrated all-in-one tooling solutions where the main benefit to the user is derived from only one part of it (because users can now mix and match as they see fit, and bundling becomes harder).
As someone who has worked on a Java language server [1] on-and-off for 2.5 years, for fun, I disagree that tooling is not fun :)
[1] https://github.com/georgewfraser/vscode-javac
You can generalize this by saying that as production values go up and user expectations rise, it takes more work and a larger organization with more people to succeed.
This is true of programming languages, but just as true of movies, games, mobile apps, and so on. It's the result of competition in a creative endeavor where it's possible to scale by adding more people.
One way to avoid this is to come up with a different "indy" aesthetic where traditional production values aren't so important. Another way is to figure out how to build on the work of others instead of starting from scratch.
I don't see how it stands to reason that this is a function of large corporations. Rust has been further along this path given their relative ages and Mozilla is no where near the size or scope of Google. Likewise with a number of other languages that have official or otherwise LSs. Same story with Cargo out of the gate, vs Go modules after nearly 5 years of pain without them, etc.
Edit: I sound more negative than I meant. I am excited about this. It was cool that Sourcegraph and the VS Code extension pioneered here, but it's exciting to have official support and a place for everyone to rally for improvements. I look forward to seeing this used in things like Theia, VS Code, etc.
The reason Go took so long with modules is another function of large corporations. Google doesn’t use modules internally (they have the monorepo) so having a module system wasn’t important.
Rust may have been further along on this path, but in the grand scheme of things Go 1.1 feels far more “complete” than Rust 2015 - and a large part of that is Go’s huge standard library that Google could fund to develop.
Rust has a lot of great tooling. Cargo is top notch. The RLS implementation is not good yet though, and there is no good news in the near term for it.
There's a thread on rust internals where planning is taking place for a long-term strategy for RLS. It looks like end of 2019 might be a realistic time frame for a compiler-backed RLS. The good news is that it looks like this work might have some significant benefits for incremental compile times too :)
> Cargo is top notch.
I agree Cargo is good but there are some issues that bug me (e.g. no support for docker layers with cargo --dependencies-only, xargo is not merged in, is there a support for custom repositories yet?).
Kinda sorta for custom repos - you can use the replace option with a proxy like artifactory, but it isn't the real deal yet.
> Here’s how languages backed by large corporations win. They have the resources and the optics to know why these non-fun features are important.
Getting started on an official autocompletion story 6+ years in, is a demonstration of this corporate-backed might?
Something like C# and VisualStudio going hand-in-hand from the outset is a demonstration of corporate-backed tooling might.
Microsoft has always been very supportive of the developers in all the technologies they put on the market (Remember the "Developers!" from Ballmer?). They have the expertise in tools and company long-term requirements. Now that they are combining all that with a more open approach, we get Visual Studio Code, the Language Server Protocol, the buying of GitHub, etc. Interesting times for the developers!
Microsoft is perhaps the very best example of this.
This is ironic, in that tooling is exactly what large mainframe systems had early on. With extensive tooling can come lack of agility, which can be its own death sentence.
That said, an advantage of being backed by a large corporation is mind share. Having corporate spending on effectively propaganda helps. A lot.
>With extensive tooling can come lack of agility, which can be its own death sentence.
What do you mean by this? Without tooling you can get stuck in the muck doing everything by hand. Tools enable you to do more with less, so I don't really understand what you are getting at here.
I think he means that it's harder to grow/innovate with the language, not that it makes coding _in_ the language harder.
Indeed I did.
Since when is tooling non-fun? This depends on a lot of factors.
Exactly. Hell, I consider tooling to be some of the most fun (and often difficult) things I work on in my free time.
If I can make something easier for my normal dev environment I see huge gains ~40 hours a week. That's pretty damn awesome and flashy to me. It's definitely not easy, but imo it's well worth my time.
I'd say tooling is so much fun it derails most of my pet projects. This is why there are 500 different static site generators produced a week.
I agree that tooling can be fun. What probably many more people would agree is non-fun is good documentation.
I actually prefer to write dev tools. Implementing my product manager next feature will always be less satisfying than helping a co-worker.
I'll be more than happy if someone pays me to implement language server.
Google does hire for these sorts of roles.
You're saying this as if their package manager isn't still garbage.
It's not?
they are fixing that too
They have been “fixing that” since like 2012.
Possibly dumb question: what is a language server in this context and what’s it used for outside of the AST package?
It's used by editors like Visual Studio Code to provide intellisense, autocomplete, breakpoints and the like.
The idea is that instead of each editor writing it's own (incomplete) Python or Go plugin, someone writes a language server which is then used (and improved) by everybody.
From the official website [1][2][3]
> The Language Server protocol is used between a tool (the client) and a language smartness
> provider (the server) to integrate features like auto complete, go to definition, find all
> references and alike into the tool. The LSP was created by Microsoft to define a common
> language for programming language analyzers to speak. Today, several companies have come
> together to support its growth, including Codenvy, Red Hat, and Sourcegraph, and the protocol
> is becoming supported by a rapidly growing list of editor and language communities.
[1] https://langserver.org
[2] https://news.ycombinator.com/item?id=18041419
[3] https://microsoft.github.io/language-server-protocol/specifi...
Look at https://microsoft.github.io/language-server-protocol/ for more info. It's a standard interface IDEs can use for things such as autocomplete rather than every IDE having to reinvent the wheel.
To add to this: the beauty of the protocol is that it uses JSON over stdin/stdout for communication. So if your language can make console apps, you can write a language server in it that works with any compliant editor/IDE - no FFI necessary, no need to learn how to author extensions for that editor etc.
And, conversely, if your editor/IDE can fork a child process and redirect its standard streams, you can support any language server and not care about how it's implemented.
I think this more than anything else has contributed to its rapid adoption, especially on Unix-likes.
Why was this downvoted? Is it not factual? If it's not, it would be far more helpful if it would be pointed out.
If it's implemented as stated by int_19h then it's my favourite interface - stdio instead of FFI. So one could implement a server even in Brainfuck.
The protocol itself is transport-agnostic - it just needs a bidirectional stream, and the spec doesn't define what it is. In practice, both stdin/stdout and sockets are used - the latter usually for language servers that are "heavyweight", such that it is advantageous to spin up the server once, "prime" it (let it do any indexing etc), and then reuse it between concurrent sessions. The standard streams are usually the default choice for simplicity reasons, though.
Implementing a server in Brainfuck is theoretically possible, but only if you ignore the heat death of the universe as the upper time boundary. ~
Interesting, I wonder if Vim is able to use this protocol as well.
https://github.com/prabirshrestha/vim-lsp
https://github.com/autozimu/LanguageClient-neovim
Look up vim lsp plugins.
Also, I wanted to ask: how does this project compares to https://github.com/mdempsky/gocode?
Go code doesn’t work with go 1.11 or at least I can’t get it to work.
mdempsky's fork of gocode should work with Go 1.11... and rstambler's fork of that fork (https://github.com/stamblerre/gocode) should work with modules.
* edit: mdempksy's fork should work with Go 1.11 used outside of a module or with GO111MODULE=off; Rebecca's fork adds support for modules on top of that
It doesn't work half the time.
I'm not sure I fully understand the provenance of the code here, but I could just be being a bit thick. The files in the change request show the copyright as "2018 The Go Authors", but some of the commentary hints that this is code donated from the Sourcegraph implementation. If this is the case, does that mean Sourcegraph has transferred the copyright to the Go authors?
I’m going to guess this is in response to many community supported dev tools breaking in v1.11 when modules are turned on. Specifically tools like gocode, godef, and gogetdoc.
Does anyone know if this is introducing more features than the existing Go LSP[1]? I'm all in favor of tooling support - I'm just curious if this actually adds anything to those of us already using a Go LSP. Thoughts?
[1]: https://github.com/sourcegraph/go-langserver
Sourcegraph CEO here (and author of a comment in the link + contributor to https://github.com/sourcegraph/go-langserver). The idea is that with a Go language server becoming a core part of Go, it will have a lot more resources invested into it and it will surpass where https://github.com/sourcegraph/go-langserver is now.
If I'm reading the linked article right this is about moving exactly the go language server you are referring to into the Google maintained set of go core libraries and tools. So this first is independent from which features it supports. Now as it is open source, you bet both Google and sourcegraph will continue adding features as appropriate.
I don't think this adds anything currently -- but the comment this post links to is by a Sourcegraph engineer :)
It's interesting that Jetbrains refuses to leverage those language servers and writes their own parsers for every supported language.
My understanding is that at least for the most popular languages, they were already doing more than the language server protocol would allow, so switching to language servers would be a downgrade.
That's their specialty, but I'm not sure it's all that relevant for users who prefer lightweight editors.
WebStorm definitely runs the TypeScript language service, but I believe they also have their own parser, completions, etc.
I think JetBrains is in a weird position. If they strictly rely on the official language server for everything, then they're not able to innovate or integrate with the many language-aware JetBrains features that go beyond what language servers typically do. On the other hand, the language server will always be a better source of truth for the basics since it's implemented by the language authors, and it requires no maintenance by JetBrains. My impression is they use a careful combination of both, but ideally it would be a unified implementation rather than two redundant implementations running side-by-side in different processes.
Why can they not extend the language server to meet their needs? Or if such feature is found useful generally for other editors too, then the language server should implement that than keep Jetbrains having some partial hacks on top of it.
The TypeScript language service is indeed extensible, but keep in mind that extensibility points can only get you so far, and JetBrains has spent the last 20 years putting a massive amount of work into a JVM-based code intelligence system. They can't just port all that to Node.js (and Go and every other language/runtime where language servers run).
Of course it would be great if the official TS language service included every imaginable IDE feature, but realistically it will always lag behind a team focused on exploring new ideas. JetBrains is going to want to experiment with new language-aware features, iterate on some of them, and throw out the ones that they aren't happy with. An open source tool shared by all editors isn't the right environment for that sort of experimentation. (They could fork the language service, though, which has its downsides but maybe is most reasonable here.)
the language server protocol doesnt support everything jetbrains does.
they can extend the server as much as they want, if the protocol doesn't support the additions, they'll be useless features unless they'll implement protocol extension, which could potentially be incompatible with the original... and they're back to the starting proposition.
I don't know if they "refuse" to leverage it. They have been building IDEs for a long time and have a toolchain for building parsers for an IDE. It would make sense for them to just use that. Besides, they were building the Go IDE before the LSP was mature.
very few languages implement the RLS well. For instance Rust does it, but its incomplete and very slow. I bought a quieter CPU fan specifically because of it!
It's more a function of language complexity than anything else. LSP for something like C, Java or TypeScript is not that hard, because the language is fairly straightforward to parse, and not ambiguous. But something like C++ or Rust, where you have to deal with templates/generics/traits/... and type inference starts getting gnarly. And something like JS or Python is hard because there are no static types at all, and you have to do inference.
But LSP itself doesn't really make it easier or harder. If you look at other code completion implementations (in IDEs that don't use LSP), they're usually at a comparable level, if they exist.
Have you used RLS recently? It’s improved quite a bit for me
Good. Great even. If the implementation is top notch, mantained and from the 1st party, why the hell not, but most of the times, language server implementations are half assed, slow, "kinda" working messes that are all but useless in the long run. The PHP language server implementations are a great example of this. A buch of broken, incomplete, underperforming projects maintained by a single person (or straight up abandoned). Compare that with Phpstorm and you have just compared a rusty tricycle with a spaceship.
Your "insightful" terminal comment is dead.
Terminals are a highly productive environment ignored at your peril. Can only tell you GUI-only folks are not well regarded in the industry.
Every language has support ... except the ones that don’t.
https://langserver.org/
I wish every language would do this.
Cool! We wrapped the VS Code extension in the protocol to support it in gitpod.io. See here https://github.com/theia-ide/go-language-server
Would be really awesome to use something maintained by the people who know Go best :)
Curious about language servers (especially those implementing LSP): how do they achieve low-latency responses?
My understanding is that JSON-RPC is used, which is relatively high overhead. Are there lighter weight alternatives? (gRPC?)
Because they're typically running locally they're quick, or at least as quick as your hardware will allow. The json-related latency is typically dwarfed by the latency from actually doing the functionality (parsing the code for code completion, etc).
So the real work is in having a good story for incremental compilation and a generally quick and robust compile toolchain. I guess that Go is in a good position here ...
The key word is "relatively". You're comparing it to protocols that are designed to process much heavier loads than what typing generates. And don't forget that processing is usually asynchronous, and can be spun off to a separate thread, so you get one just for protocol handling.
The xi editor uses json to communicate with its backend. The author wrote about negligible overhead
Good idea. Languages like Haskell benefit by having a language server that Emacs, VisualCode, etc. can all use. It just make good sense to do the work once and let all editors and IDEs use it.
This is great news. Hopefully we will see others follow.
2 unrelated questions. (tl;dr at bottom)
The two major advantages of Go are its clear, unambiguous and explicit syntax (unfortunately ugly) and very small (minimal) feature set, so that it is possible to reason about anything without any high context.The former means it is possible to pick up Go in a day or two and the latter means that there are no real tricks after a few months. That's it.
All this has been done in an excellent way. There is no class system. Everything is simple. Explicit typing works.
These design trade-offs have led to a language that is VERY easy to go back to after working in any other language. Almost all simple syntax mistakes by the programmer are caught before something successfully compiles. Once a programmer is used to it, which happens quickly, Go is the only language people write pages of code in that compiles on the first pass and does everything they hoped. "Less is exponentially more."
Due to the history of Go it was meant to be a systems language. It didn't really succeed at replacing C anywhere, except the web server: I consider Go like a web-safe C. (In no small part due to Google's work meeting its own needs.) What I don't understand is that since speed and being a systems language were explicit design goals, why the Go syntax is not suitable for tiny, embedded microcontrollers.If the answer is something about garbage collection, this doesn't really make sense for me, as good algorithms should scale from 0 upward. I don't see why there should be a huge jump from "not running Go yet" to "now you're running a Go runtime" that suddenly requires a large amount of free memory. Here is a post about this:
https://medium.com/samsara-engineering/running-go-on-low-mem...
It seems this is high.
I don't understand why Go doesn't take its core, primary strengths outlined in the "background" section above and expand it into the lowest of the low-end devices.
Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue? Go was designed as a compiled systems language, to replace C. Not as a scripting language to replace Python.
Since explicit (if very ugly) syntax that keeps people from making mistakes, and whose specifications can be kept in mind all at once, is a strength of the language and syntax, why can't I develop desktop GUI's in Go? It is a fast, compiled language.You might say the answer to that is that a windowing system needs a class system but I disagree - that can't possibly be any more true than the idea that a web app needs a class system. Clearly people make web apps in Go without a class hierarchy.
Nobody makes desktop apps (I mean the class of applications that web browsers, IDE's, photo editing software, basically everything on a list like this - https://helpdeskgeek.com/free-tools-review/best-freeware-pro... ) in Go.
Of course, when it debuted you could say, "That's just not what the team focused on", since it was backed by Google they needed it on the server. But at some point you'd expect first someone to make a make-do GUI toolkit, then other people to add on and make something better, and by now it should be an obvious language for making a Winzip or Winrar competitor. Or an antivirus. Or an Audacity or VLC-type program. Or an OpenOffice clone.
But nobody does these things. Any of these things.
Why not? The simple language has nothing missing that languages which are used for these applications have.
I don't buy the idea of the ecosystem entirely lacking these.
Why don't people make GUI stuff in Go? I don't understand why it shouldn't be an obvious language choice. I just wouldn't reach for Go when it comes to scripting a GUI application. Why not?
1. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for embedded microcontroller applications in 2018.2. I would like to know what it would take to make the simple specifications behind Go, which are protected by knights in shining armor, an obvious choice for desktop applications in 2018.
I think these are good questions, even if they are off topic.
1. Why doesn’t Go seem to support embedded development. Off the top of my head there is a moderely successful embedded framework, gobot. The reason I don’t think it’s more popular is the runtime. Having any runtime at all tends to be poisonous and running Go without a runtime is difficult because you can’t manage your own memory in Go. Sure you could do a ton of work in making it work, but your effort is better spent just writing your project in Rust.
2. Why doesn’t Go support desktop? This is a case where I just don’t think anyways cares anymore about having good desktop support. You rarely see native desktop applications in anything other than C++ and C#. There are some UI toolkits out there but no one uses them, and something like Electron is used instead. Windows and OS X only give first class support to their blessed languages (C++, .NET, Objective C) and every other language you are out of luck.
I'm curious what you think the primary feature is that makes a Go runtime have to be necessarily large? (If writing a new Go runtime against the specification.)
You mention the need for a runtime, but I don't see that this makes it necessarily huge. For comparison, I'd like to think about using a TinyC compiler[1], called TCC, as a complete runtime to compare against since:
"
* TinyCC (aka TCC) is a small but hyper fast C compiler. Unlike other C compilers, it is meant to be self-relying: you do not need an external assembler or linker because TCC does that for you.
* TCC compiles so fast that even for big projects Makefiles may not be necessary.
* TCC not only supports ANSI C, but also most of the new ISO C99 standard and many GNUC extensions including inline assembly.
"
Now it turns out that the above fits in 100 KB and implements "ANSI C", which for the C89 version is 552 pages or so[2]. The Go specification is 73 pages or so in the same font. (So 13% of the size.) I also think that due to its design the Go language specification and syntax has fewer corner cases.
So clearly there is some "huge" feature that is really holding things back from running on a resource-constrained microcontroller in a stripped-down runtime. What is it?
To see where 100 KB fits (by analogy with TCC above), I looked up some microcontrollers. I first found a popular microcontroller for hobbyists, the ATmega2560 which runs the Arduino Mega. That microcontroller has 56KB ISP flash memory, 8KB SRAM. That RAM is not enough if we take the TinyC executable of around 100 KB as how much is needed. This microcontroller also only costs $6.5 in quantity 1 (unless I misread.)
Moving up a bit in price so we can get at least 128 KB of RAM, a popular choice is something like the AT32UC3A3256[3] which costs $31.25 in an evaluation kit (that gives you an additional 8 MB of RAM), and as far as shipping just the MCU (with its 128KB of RAM only), costs $12 in quantity one. That is a 32-bit device unlike the cheaper Atmega2560 I first mentioned (which is an 8-bit device) so should be easier for people to implement the standard against. The specifications are: 84 MHz, 256 kB program size, 128 kB RAM size.
Why can TCC be 100 KB executable size but independently implementing the Go specification can't?
As for why someone would want to, sure, you're going to be quite constrained, but if someone already knows Go, which had the design goal of being a general-purpose language, why can't they use its syntax for their microcontroller projects?
So what essential feature in the Go specification makes this an absolute non-starter? it's a super small standard.
(As for your second response, about "You rarely see native desktop applications in anything other than C++ ", I read what you wrote but I don't see why the Go ecosystem isn't slowly catching up anyway. It's a great language syntax.)
---
[1] https://bellard.org/tcc/ and https://bellard.org/tcc/tcc-doc.html
[2] found a draft here - http://www.open-std.org/JTC1/sc22/wg14/www/docs/n1256.pdf
[3] I found this recommendation here: https://electronics.stackexchange.com/questions/192583/is-th...
> So clearly there is some "huge" feature that is really holding things back from running on a resource-constrained microcontroller in a stripped-down runtime. What is it?
Garbage collection and goroutines. You could probably strip back the implementations a bit but you can't get rid of gc - there is no explicit way of freeing memory. Possibly you could ban the go keyword and run on a single thread, although you'd have to deal with parts of the stdlib that use it.
Java and Oberon fit easily in a ARM Cortex M, which some embedded companies are happily selling, there is no reason why a Go implementation could not exist as well.
It looks like Go implementation exists for ARM Cortex M (based on transpiling Go to C):
https://github.com/ziutek/emgo/blob/master/README.md
https://github.com/ziutek/emgo/issues/14#issuecomment-398670...
https://ziutek.github.io/2018/03/30/go_on_very_small_hardwar...
There is also GERT, but it is for ARMv7a like iMX6 Quad SOC and not updated since one year.
>Garbage collection and goroutines
Do you think the specification is "correct" for a compiled language suitable for embedding?
The word "garbage" is in the specification a single time, in the introduction which says "garbage-collected". I can't see any constrained implementation details thereafter - see for yourself at where I might have missed it:
https://golang.org/ref/spec
The gouroutine section (under the Go keyword) is three paragraphs: "A "go" statement starts the execution of a function call as an independent concurrent thread of control, or goroutine, within the same address space." and "The function value and parameters are evaluated as usual in the calling goroutine, but unlike with a regular call, program execution does not wait for the invoked function to complete. Instead, the function begins executing independently in a new goroutine. When the function terminates, its goroutine also terminates. If the function has any return values, they are discarded when the function completes. "
This does not seem to require the typical accoutrements of the Go compiler. If this is the complete verbiage specifying the behavior of a Goroutine, I could imagine a compliant implementation being lighter-weight than the official Go compiler.
Returning to the idea of GC: the word "memory" appears in the standard 4 times: once mentioning initialization of variables; once on the section on slices ("The built-in functions append and copy assist in common slice operations. For both functions, the result is independent of whether the memory referenced by the arguments overlaps. "); and twice in the last section on system considerations, including light MAY language in "Computer architectures may require memory addresses to be aligned; that is, for addresses of a variable to be a multiple of a factor, the variable's type's alignment. The function Alignof takes an expression denoting a variable of any type and returns the alignment of the (type of the) variable in bytes." and finally under Size and Alignment guarantees, which makes no reference to GC.
If this document is the complete specification, then it seems that the implementer has very wide leeway for how to implement the details of memory management via garbage collection. There doesn't seem to be much language limited behavior.
What do you think?
(In this comment I consider "Go" the language as anything that complies with the Go specification.)
To point 2) I fully agree, Go should be a great choice for desktop applications. One match I think is especially good, that is Gtk based development with Go. Gtk is one of the few UI toolkits which has been designed around a pure C API, so Go is a great match with it. Also, with Gtk the Go applications are reasonably portable to Linux, macOS and Window. I have used Go+Gtk and found it a pleseant experience.
<>
Firstly, wow! Thanks. Feedback on draft #2.5:
You removed too much praise. Go's syntax is amazing. Can you put that part back in?
(Maybe you removed it because I also called it ugly - as opposed to a beautiful syntax like Python's - but that was not a design goal and doesn't bother me. Nobody meant for Go's syntax to be beautiful.) The summary you got down to "I appreciate Go since" is too weak.
Its syntax is the best of any language on planet Earth, which is why people can write pages of Go that work on first compilation and always do exactly what they imagine. (The second fact that leads to this is that its specifications and feature set is very small, so that it can be kept in mind by the programmer entirely.)
So add stronger praise. If not for such strong praise, why would I want to write everything from desktop applications to microcontroller code in Go, and not any other syntax?
I also have a very strong impression that its syntax and specifications are appropriate for both memory-sensitive applications and for desktop applications. (Not really a question). It is due to this strong impression that I want to know what the impediments are. (Maybe this was not clear.)
I can see no reason why a Go executable can't be made targeting 100 KB of RAM to do its GC and everything else. I also see no reason why bindings against QT or other GUI frameworks aren't made in the Go language syntax - this is a mystery to me. There must be some impediment.
What is it?
Can you reply with another draft that incorporates these?
I greatly appreciate the thoughts you added to the draft. I think Go is the best language syntax for general programming in any environment from a microcontroller with 150 KB of RAM to a major, 2 Gigabyte desktop behemoth's scripting language. I don't think the lack of class system is an issue (which is the only thing that comes to mind for me.)
So you need much stronger praise, please. Go is beautiful (despite its visually ugly syntax) and its use for web apps shows/proves its total appropriateness to all applications.
I want to understand the impediments. It should be much farther along to replacing C and C++.
Also:
>But I worry that Go is moving away from this mantra over the initial specification
I don't have such a worry. I guess you could say:
"But my impression is that the Go compiler leaves a lot of the huge benefit that is still in its specification: that is, a safer replacement for C and C++. It is perfectly viable as a replacement for nearly all C and C++ language work."
I'd like to understand why it isn't doing so yet and what it will take to get there. Hope this helps clarify my thinking.
Turn up the praise.
Not going to edit the above but people clearly hate it (many downvotes without minutes of posting).
At the moment I think most Go programmers who also program tiny microcontrollers with a few hundred KB or up to a couple of MB of RAM, do so in languages like C or Rust, not in Go. Why not? It's a fair question.
For those of you reading this on a laptop or desktop, how many desktop applications are you all running at the moment or have as an installed application? How many are written in Go? Why so few or none?
I mean desktop applications like Audacity for audio, or VLC, or Open Office, or your web browser, and so forth.
The downvotes came pretty fast so you clearly have some ideas. I'd like to hear them.
>people clearly hate it
Maybe because it's overlong, almost completely unrelated to the article, and might just be flamebait?
But I'll toss in a few cents anyway.
>Why can't I compile a Go program targeting a microcontroller with 2-8 MB of RAM without issue?
Microcontroller toolchains are pretty controller-specific. It takes work to make things compile for them. There are other languages that fare better. Additionally, one of the more useful features of Go (concurrent goroutines) would simply be unavailable (and even with concurrency disabled, you still get unpredictable latencies from deciding which task should run when, the GC, etc).
Just use C.
>why can't I develop desktop GUI's in Go
You can. https://github.com/andlabs/ui is an example of how you can (and I'm sure it's not the only one.)
>Why don't people make GUI stuff in Go?
It's common to write apps using whatever the OS provides or using well-known libraries, so that's something Go's offerings don't offer. It's so much work already to build a GUI app that most people doing so probably don't want to concurrently write a GUI library too. I don't think there's anything stopping you, though.
Next time, avoid writing so much to ask two (three?) simple questions. You come off as having an attitude when you use phrases like "Not going to edit the above", "knights in shining armor", and so on. You also build up and tear down many bales worth of strawmen, which is pointless. It would have been a better use of your time to actually look into either of those fields and discover the answers for yourself.
Thanks for your responses, which I read carefully. I hope we can get a bit more out of this conversation.
"Just use C" is odd to me given that Go was originally developed to be an alternative to C (a "systems language") - this was an explicit design goal, and mentioned in the first main sentence of its specification[1]: "Go is a general-purpose language designed with systems programming in mind."
So when you then glance at the adoption for those uses, you would come to the (wrong) conclusion that it must have failed to meet its original design goal of being better for those general use cases. (Or people would be using it). According to its designers, C and C++ programmers don't transition to Go (as its designers had thought they would), but Python programmers do.
So I'd like to understand what it lacks that is holding that back. I see it as clearly superior. Looking at the design, it looks like it met its design goal. So I see it as successful in its bid to be design that is a viable replacement for C as a "systems language", and then despite this successful design now nobody uses it as a systems language.
It would be as though someone set out to make a better bicycle wheel, succeeded, and then nobody uses it as a bicycle wheel (despite its being a better bicycle wheel), while meanwhile the design is being used to haul thousands of tons of freight per year (server microservices). I'm asking, "so why don't we use it as bicycle wheels" and am told "just use a wheel". But I want to understand why, because it seems to me it met its design goal of being a better bicycle wheel.
For the second statement, your statement about "common to write apps using whatever the OS provides or using well-known libraries," makes sense and explains why these libraries didn't exist when Go was made. I don't understand why they are still not a big part of the ecosystem nearly a decade later.
The "knights in shining armor" is a serious statement: the biggest benefit of the Go language is its gatekeepers keeping things out of it, and its extremely orthogonal, tiny syntax, which can be kept in mind along with state. The results of code can be reasoned about without a lot of gotchas. I am therefore wondering what, if anything, these gatekeepers are keeping out which prevents the two parts of the ecosystem which I mentioned, from flourishing.
I don't think Go programmers who are also C and C++ programmers who are used to writing for embedded microcontrollers or for Windows in C and C++ respectively, require deep (or any) changes to the language before as a language it would be suitable to those two cases. You haven't mentioned any.
So I still welcome more viewpoints. I don't see why Go can't replace C and C++ applications across all of the use cases I mentioned, with minimal first-party support by the language developers and maintainers. Doesn't Go also have alternative Go compilers based on the specification?
I did a Google search for alternative Go compilers and the top result is from 2014: https://groups.google.com/forum/#!topic/golang-nuts/EwDS15Ew...
As a followup, why is this the case? I mean, the Specification is quite clear, and it's a tiny language, so in the past 9 years I don't see why there aren't at least a few dozen independent implementations. Is it just not enough time, and we should wait a few decades? Is the issue that Google doesn't need Go for its desktop or mobile development stuff, and doens't need it for microcontrollers, and Go is not really an "independent" entity that is quite independent of Google as a company?
Basically I am wondering why Go in 2018 isn't where C and C++ were by 1998, for example. Despite your detailed answers I don't see the missing pieces.
Go as a syntax is better than C or C++ as a syntax. Its syntax and small and stable language features set are Go's primary advantages, which allows it run on the metal using clearly written standard Go libraries to serve web apps. To me, a microcontroller program running with next to no memory just has to not leak memory or crash, unlike Go running microservices, which have to withstand highly targeted and sophisticated attacks.
Basically, given how I see Go (which is the reason for my very lengthy introduction, so that people could correct me if I'm wrong) I don't see why the language is not very far along where C and C++ blossomed as languages for embedded microcontrollers, and for Desktop applications.
In a way you could say I don't really get what's preventing this. I can't think of any showstoppers.
[1] Go language specification: https://golang.org/ref/spec
I tried to find a succinct question to respond to for your first section, and couldn't. My answer is the same as before: no one has bothered to write a Go runtime that can run on microcontrollers. This is because Go does not do anything fundamentally different from C at that level, except things that are less suitable for low-memory and low-CPU environments; and because it takes a lot of work to write a compiler and runtime for these devices. Note that there are very few compilers for each individual microcontroller. They generally only have one or two.
>I see it as clearly superior.
You are not looking hard enough. Go has flaws, as does any language or tool. It is almost a rule of thumb that not understanding the flaws of a tool is a sign of inexperience with it or alternatives to it. I think if you actually tried to learn and use C and C++, you might better understand the difference.
>I don't see why Go can't replace C and C++ applications
My answer is the same as before: it can, except that C/C++ apps already have that territory, and Go doesn't provide a fundamentally different experience except in lacking libraries for most GUI applications. There are still places it shines, though. For example, it is doing a fine job of replacing C/C++ for cloud orchestration tools; static linking is important to those users, and Go provides it easily.
>Basically I am wondering why Go in 2018 isn't where C and C++ were by 1998, for example.
This is a mathematical error. The C programming language was developed in 1972, and C++ piggy-backed off of C's ecosystem very effectively.
>Go as a syntax is better than C or C++ as a syntax
I don't think I can come up with a better example of a "skin-deep" comparison, nor do I think anyone has actually argued for C/C++'s syntax being actually good.
>stable language features
Didn't Go 1.11 break all of the community's tools? Isn't that why the story we're posting on the comments on exists?
>To me, a microcontroller program running with next to no memory just has to not leak memory or crash,
Some microcontroller programs need to have very low latencies, or the physical things they are connected to will break (the GC defeats this). Some microcontroller programs are only allowed a couple of KB of RAM (Go can't handle that). Most microcontroller programs are compiled without a standard library (and Go would have to provide its own impls of basic Go stuff in order to function). Some microcontroller programs never allocate memory, cannot allocate memory, or must allocate all memory up front. And so on. You can solve these things by having more expensive and fault-tolerant gear, or buying more RAM, or paying people to write more Go compilers/runtimes, etc., but these things get expensive fast; that is often incompatible with the design goals of what microcontrollers are used for. You're welcome to write Go and slap it on a Raspberry Pi, though.
>In a way you could say I don't really get what's preventing this.
Try writing a microcontroller program and GUI app, both in C or C++ and in Go. I very strongly doubt you will succeed faster in Go than in C/C++, in either kind of project, because of multiple factors: ecosystem, communities, example code, language features, existing toolchains and compilers... But if you don't believe me, try it out for yourself.
I read this several times hours apart. Lots of information, thanks. I don't have many questions except this:
"Some microcontroller programs are only allowed a couple of KB of RAM (Go can't handle that)."
What in here...
https://golang.org/ref/spec
...can't you possibly implement in only a couple of KB of RAM? (For what required feature or features can we say for sure that a couple of KB of RAM is insufficient to implement as specified?)
As it turns out, the only mention of the feature is in that link's introduction. The garbage collector (https://github.com/golang/go/blob/master/src/runtime/mgc.go) makes many references to 4KB pages and 1MB arenas. This (the code and the user of the technique) would need some serious reconsideration for small devices.
Isn't the specification the actual language, and Google's compiler just one implementation? (That's true for C/C++.)
Is the specification underspecified or something? We're not talking about a syntactically huge language...
I don't downvote you but ... No one hate your post. It's just an huge wall of text unrelated to the current post. It seems not the good place or the good moment to discuss this. You can maybe try to do an "Ask HN" for open question like this.
I think it's a great place to ask this. The most fantastic people to answer the question are going to be reading a post like this, and what we're reading about (adding a Go language server to core) is a huge and language-specialized feature for core. I look forward to reading people's answers to my questions.
I thought the downvotes were because of something I got very wrong. Anyway I hope to learn from responses even if I got something wrong.
Didn't downvote your post above, so that wasn't me.
That aside, there are some resources around for this though I've not (yet) tried any of them:
* http://embd.kidoman.io - Go Embedded Programming Framework. Seems to support Raspberry Pi and BeagleBone Black.
* https://gobot.io - Robotics more than embedded, but there seems like some overlap. :)
---
With the "Why no desktop apps in Go?" question, for me it's because I'd only do so using something like Qt (cross platform).
While there have been several attempts to get usable Go bindings happening for Qt, none of them seem both really well done + long term stable.
The best developed Qt bindings so far are https://github.com/therecipe/qt. But that seems to have tapered off and the developer doesn't seem interested in making it into a team project (eg long term viability). That's my impression anyway, I could be wrong. :)