The "unconst" business is... original, but it seems backwards to me. If I understand correctly, applying the Fibonacci function to a constant will compile a new version of the function, with the loop fully unrolled. And to avoid this, you have to mark some variable that is not the function argument but is initialized to a constant as "unconst". That just seems like a bizzarre source of confusion.
I'm all for aggressive user-controlled loop unrolling and partial evaluation, but it should be user-controlled, not "I'll just always do it behind your back and sometimes surprise you with an obscure error message that requires a non-obvious fix and will make your function non-optimizable even in cases where you would want to request it".
Also, yay for named loops, but the syntax is weird. To me,
let foo ... = ...
if (bar)
...
Doesn't look like one loop but two unrelated statements. I understand that it's the underlying label/goto abstraction leaking through, but that looks like a backwards jump (haha) in time.
Regarding "unconst", we need to have a discussion. There are ways to make it implicit, namely by tying the return continuation of `if` to the constness of the conditional, but then one loses the ability to specialize the return continuation for constant results.
Named `let` is the most primitive form of loop, suitable for use in runtime and compile time (constexpr-style) context. There are more convenient, less versatile and compiler-friendly versions like `for ... in` and `while`. It is possible to design your own loop structures, and even build an auto-unconsting `if`, so I was hoping that the community would innovate here within the language.
Partial evaluation is user-controlled. The rules are deterministic: the system guarantees that constant expressions are always folded. The error message would not exist if we could solve the halting problem. There is no way to tell in every instance if a loop or recursion ever terminates, so I went for the simplest rule, which is that if a label is reentered the 32th time, we abort compilation. It's a cautious limit because constant folding is interpretation (slow), and I don't think that programmers want big loops to unroll. This can and should be discussed. I certainly don't have all the answers.
> I understand that it's the underlying label/goto abstraction leaking through
Scopes' abstractions aren't leaking, they are offering their innards to you. You can generate and inspect IL from within the language. Everything has been made so that you can build your own abstractions on top of them, which is only possible when the internals aren't hidden. The philosophy is that a compiler is a servant, not a framework.
> I don't think that programmers want big loops to unroll.
Right, well, it depends. That's why I think unconst should be the default and unrolling should be requested explicitly by using something like
let fib_2000 = specialize (fib 2000)
where specialize is a language built-in that triggers the unrolling. And then, if the user really does want you to specialize fib 2147483647, it would honor that too, without your arbitrary unrolling cutoff. And if it takes too long, well, the user wanted it that way. That is user control. Arbitrary hidden cutoffs are not.
unconst can't be the default because there is no way to re-const a value once it's been turned into an unknown. The most "automagic" behavior is to tie branch conditionals to their exit values (there's a helper function called tie-const that does this for the manual case). If the conditional is unknown at compile time, then the continuations that exit the branch will mark their arguments as unknown. That also stops loops from unrolling (they'd typically unroll for i=0, but then stop). But, as I mentioned before, that completely kills constant folding at the exit points, something that is at times desirable.
Another reason loops are unrolling by default is that there are many situations where constexprs have to completely fold at compile time, and you need a guarantee that they do. A common example is handling keyed varargs. You need to be able to iterate through the keys and build local variables from that without any code being generated.
If a compile time unroll takes forever, there's practically no clean way to debug that.
On the other hand, I like unconst on first sight. A symmetrical form to do specialization would be nice too. What about making the loop unrolling configurable, whether with a compiler flag or some form like Lisp's declare.
nota that Jai has a similar concept, can't give you a link right now. It allows a function to have several instantiations. The generic (slow) function is mostly documentation, not intended to be used :)
> the compiler is designed to remain on-line at runtime so that functions can be recompiled when the need arises, and generated machine code can adapt to the instruction set present on the target machine. This also diminishes the need for a build system
Diminishing the need for a build system is nice! Any other language with such philosophy in mind?
> The Scopes compiler fundamentally differs from C++ and other traditional AOT (ahead of time) compilers, in that the compiler is designed to remain on-line at runtime so that functions can be recompiled when the need arises, and generated machine code can adapt to the instruction set present on the target machine. This also diminishes the need for a build system. Still, Scopes is not a JIT compiler. Compilation is always explicitly initiated by the user.
Many Lisp systems also have their native code compiler on-line.
It hasn't obviated the need for a build system, but this is a traditional feature of Lisp [1]. With modern security requirements, though, I'm not sure how popular this approach is going to be.
Long story short I found a commit [1] which mentioned macros. Looking at the diff suggests macros are defined using "syntax-extend" so I looked for that syntax in the current version of the code. Lo and behold [2]
Hey paniq, based on your username I'm guessing that you are the same paniq mentioned on the duangle website and thus the creator of this intriguing little language! Scopes has certainly caught my interest in the short time I've looked at it this evening, but the documentation seems a bit sparse at the moment. I do have your screencasts queued up on YouTube and the repository cloned and ready to build, but if you have any thoughts you want to share about how you've used the language, what it's good for, the direction you want to take it, if you want community contributions I would love to hear them!
Hi, glad to hear you like it. I'm using Scopes to build a game engine called Tukan, which is for an ambitious game called Nowhere that is in development hell for four years. :-) I don't really know what I want from the community to be honest. I guess I just want to collect first impressions, see what people like, what they struggle with, what things they need and what ideas they come up with.
I do think that Scopes is uniquely suited for monolithic multimedia applications like games and media authoring and that would be the direction I'd like to see it go. But there's technically no reason why it couldn't go another way.
I don't know what those are, but Scopes is designed to make it easy for users to implement their own meta-programming abstractions. It might be possible to build such mechanisms from within the language. I certainly want to hear of the instances where it is not.
It has a REPL, supports S-expressions as a special case of the syntax, has garbage collection[1] and doesn't appear to have more C-ish semantics than Scheme already has.
That code does indeed automatically delete some entries from some collection, and it does use garbage collection terminology, but it doesn't appear to be full garbage collection for the whole language. Hard to tell in the complete absence of comments and a commit message.
But http://scopes.readthedocs.io/en/latest/about.html says "The memory model is compatible to C/C++ and utilizes simple unmanaged stack and heap memory." And I do recall paniq tweeting about wanting to implement a Rust-like system for memory management, because how hard can it be? (Fortunately, it doesn't look like he ever tried it.)
In addition, https://bitbucket.org/duangle/scopes/wiki/Home contrasts "expressivity of Scheme" and "the performance and runtime model of C" as if the developer did think of these as opposites.
> That code does indeed automatically delete some entries from some collection, and it does use garbage collection terminology, but it doesn't appear to be full garbage collection for the whole language. Hard to tell in the complete absence of comments and a commit message.
It's very old code from two years ago, when the language was still being prototyped. I did have a form of GC for a while that I threw away again because I couldn't make it work right.
I recommend not taking any commits before the first tagged release seriously.
> Fortunately, it doesn't look like he ever tried it.
I tried, but I'm completely paralyzed here. As long as I can't see a straightforward way to integrate support into the typechecker, I won't write a single line.
Also, why "fortunately"?
> as if the developer did think of these as opposites
Not opposites, just not aligned with each other. The demands of Scheme's runtime model inherently conflict with realtime performance. There's nothing wrong with that, but that sets it apart from C.
With this sentence, I hoped to address Scheme users who wish they could generate faster, tightly-constrained assembly while not having to give up on hygienic macros and nested evaluation contexts. No insult was intended.
Because I think it's a lot harder (both for you and for your users) than you thought at the time, and I hoped you wouldn't waste your time (and your users' time) on it. Personally, I think GC is the way to go to keep your (programmers') sanity. If you worry about GC pauses in time-critical sections, provide for a way to turn off the GC temporarily for such sections.
You might want to mark such non-GC sections/functions specifically and enforce statically that no dynamic allocation may be triggered from within them. Kind of like a monad. No allocations, no GCs, no headaches. Simpler and more flexible than the Rust model.
> Because I think it's a lot harder (both for you and for your users) than you thought at the time, and I hoped you wouldn't waste your time (and your users' time) on it.
I believe it is possible to offer such a feature without declarative baggage. If it's not, then I don't want it ;-)
> Personally, I think GC is the way to go to keep your (programmers') sanity. If you worry about GC pauses in time-critical sections, provide for a way to turn off the GC temporarily for such sections.
It might be even possible to leverage Scopes' support for compile-time introspection to write contextual GC's from within the language. I haven't explored that yet. In games development we usually prefer to allocate in frames, stages or on custom stacks, so that's also a way to do it.
Installing a GC at the lowest level is a brutal choice that precludes many other possible ways this could go, so I'm cautious.
S-expressions are not a special case of the syntax, S-expressions are the syntax. There are two notations, one is naked, python-looking, and one is traditionally braced, lispy-looking. It is possible to enter from one notation into the other at any point, depending on what suits the situation best.
Scopes has no garbage collection. There will be garbage collection for compile time symbols at some point, but the runtime is completely manually managed. That is a big difference to Scheme, although Scheme has been a major influence to Scopes design, and features that I believe to set Lisp/Scheme apart from other languages have been adopted.
Scopes is statically typed, but type signatures are not elementary to function definitions and value declarations, as with Scheme. It is a little limited with its compile-time closures though, whose application as first class values is limited.
Tutorial and examples: http://scopes.readthedocs.io/en/latest/tutorial.html
The "unconst" business is... original, but it seems backwards to me. If I understand correctly, applying the Fibonacci function to a constant will compile a new version of the function, with the loop fully unrolled. And to avoid this, you have to mark some variable that is not the function argument but is initialized to a constant as "unconst". That just seems like a bizzarre source of confusion.
I'm all for aggressive user-controlled loop unrolling and partial evaluation, but it should be user-controlled, not "I'll just always do it behind your back and sometimes surprise you with an obscure error message that requires a non-obvious fix and will make your function non-optimizable even in cases where you would want to request it".
Also, yay for named loops, but the syntax is weird. To me,
Doesn't look like one loop but two unrelated statements. I understand that it's the underlying label/goto abstraction leaking through, but that looks like a backwards jump (haha) in time.Regarding "unconst", we need to have a discussion. There are ways to make it implicit, namely by tying the return continuation of `if` to the constness of the conditional, but then one loses the ability to specialize the return continuation for constant results.
Named `let` is the most primitive form of loop, suitable for use in runtime and compile time (constexpr-style) context. There are more convenient, less versatile and compiler-friendly versions like `for ... in` and `while`. It is possible to design your own loop structures, and even build an auto-unconsting `if`, so I was hoping that the community would innovate here within the language.
Partial evaluation is user-controlled. The rules are deterministic: the system guarantees that constant expressions are always folded. The error message would not exist if we could solve the halting problem. There is no way to tell in every instance if a loop or recursion ever terminates, so I went for the simplest rule, which is that if a label is reentered the 32th time, we abort compilation. It's a cautious limit because constant folding is interpretation (slow), and I don't think that programmers want big loops to unroll. This can and should be discussed. I certainly don't have all the answers.
> I understand that it's the underlying label/goto abstraction leaking through
Scopes' abstractions aren't leaking, they are offering their innards to you. You can generate and inspect IL from within the language. Everything has been made so that you can build your own abstractions on top of them, which is only possible when the internals aren't hidden. The philosophy is that a compiler is a servant, not a framework.
> I don't think that programmers want big loops to unroll.
Right, well, it depends. That's why I think unconst should be the default and unrolling should be requested explicitly by using something like
where specialize is a language built-in that triggers the unrolling. And then, if the user really does want you to specialize fib 2147483647, it would honor that too, without your arbitrary unrolling cutoff. And if it takes too long, well, the user wanted it that way. That is user control. Arbitrary hidden cutoffs are not.unconst can't be the default because there is no way to re-const a value once it's been turned into an unknown. The most "automagic" behavior is to tie branch conditionals to their exit values (there's a helper function called tie-const that does this for the manual case). If the conditional is unknown at compile time, then the continuations that exit the branch will mark their arguments as unknown. That also stops loops from unrolling (they'd typically unroll for i=0, but then stop). But, as I mentioned before, that completely kills constant folding at the exit points, something that is at times desirable.
Another reason loops are unrolling by default is that there are many situations where constexprs have to completely fold at compile time, and you need a guarantee that they do. A common example is handling keyed varargs. You need to be able to iterate through the keys and build local variables from that without any code being generated.
If a compile time unroll takes forever, there's practically no clean way to debug that.
On the other hand, I like unconst on first sight. A symmetrical form to do specialization would be nice too. What about making the loop unrolling configurable, whether with a compiler flag or some form like Lisp's declare.
nota that Jai has a similar concept, can't give you a link right now. It allows a function to have several instantiations. The generic (slow) function is mostly documentation, not intended to be used :)
From the intro: http://scopes.readthedocs.io/en/latest/about.html
> the compiler is designed to remain on-line at runtime so that functions can be recompiled when the need arises, and generated machine code can adapt to the instruction set present on the target machine. This also diminishes the need for a build system
Diminishing the need for a build system is nice! Any other language with such philosophy in mind?
> The Scopes compiler fundamentally differs from C++ and other traditional AOT (ahead of time) compilers, in that the compiler is designed to remain on-line at runtime so that functions can be recompiled when the need arises, and generated machine code can adapt to the instruction set present on the target machine. This also diminishes the need for a build system. Still, Scopes is not a JIT compiler. Compilation is always explicitly initiated by the user.
Many Lisp systems also have their native code compiler on-line.
It hasn't obviated the need for a build system, but this is a traditional feature of Lisp [1]. With modern security requirements, though, I'm not sure how popular this approach is going to be.
[1]: http://www.paulgraham.com/diff.html "The whole language always available"
The copy says that it has the “expressivity of Scheme,” but the documentation has no mention of macros :(
The author has published a number of screencasts and it indeed supports macros: https://www.youtube.com/watch?v=8XN93cKh7Gc&list=PLWVpbdR2y1...
They do seem to exist, based on the notes in the release history. I'll link when I'm off my phone, forgive me.
Long story short I found a commit [1] which mentioned macros. Looking at the diff suggests macros are defined using "syntax-extend" so I looked for that syntax in the current version of the code. Lo and behold [2]
[1] https://bitbucket.org/duangle/scopes/commits/ca7126d88bbd [2] https://bitbucket.org/duangle/scopes/src/1bf171c25f4047bc049...
Hey paniq, based on your username I'm guessing that you are the same paniq mentioned on the duangle website and thus the creator of this intriguing little language! Scopes has certainly caught my interest in the short time I've looked at it this evening, but the documentation seems a bit sparse at the moment. I do have your screencasts queued up on YouTube and the repository cloned and ready to build, but if you have any thoughts you want to share about how you've used the language, what it's good for, the direction you want to take it, if you want community contributions I would love to hear them!
Hi, glad to hear you like it. I'm using Scopes to build a game engine called Tukan, which is for an ambitious game called Nowhere that is in development hell for four years. :-) I don't really know what I want from the community to be honest. I guess I just want to collect first impressions, see what people like, what they struggle with, what things they need and what ideas they come up with.
I do think that Scopes is uniquely suited for monolithic multimedia applications like games and media authoring and that would be the direction I'd like to see it go. But there's technically no reason why it couldn't go another way.
Is there support for effects or is there something like a concurrency monad ?
I don't know what those are, but Scopes is designed to make it easy for users to implement their own meta-programming abstractions. It might be possible to build such mechanisms from within the language. I certainly want to hear of the instances where it is not.
https://github.com/yallop/effects-bibliography
Anyway, the type of the print "hello world" expression would explain a lot.
So this is a Scheme with pythonic syntax?
It doesn't seem to be very close to scheme. It's statically typed, AOT compiled, not garbage collected, and has C-ish semantics.
It has a REPL, supports S-expressions as a special case of the syntax, has garbage collection[1] and doesn't appear to have more C-ish semantics than Scheme already has.
[1] https://github.com/duangle/scopes/commit/13255e0ae7ab76e070c...
That code does indeed automatically delete some entries from some collection, and it does use garbage collection terminology, but it doesn't appear to be full garbage collection for the whole language. Hard to tell in the complete absence of comments and a commit message.
But http://scopes.readthedocs.io/en/latest/about.html says "The memory model is compatible to C/C++ and utilizes simple unmanaged stack and heap memory." And I do recall paniq tweeting about wanting to implement a Rust-like system for memory management, because how hard can it be? (Fortunately, it doesn't look like he ever tried it.)
In addition, https://bitbucket.org/duangle/scopes/wiki/Home contrasts "expressivity of Scheme" and "the performance and runtime model of C" as if the developer did think of these as opposites.
> That code does indeed automatically delete some entries from some collection, and it does use garbage collection terminology, but it doesn't appear to be full garbage collection for the whole language. Hard to tell in the complete absence of comments and a commit message.
It's very old code from two years ago, when the language was still being prototyped. I did have a form of GC for a while that I threw away again because I couldn't make it work right.
I recommend not taking any commits before the first tagged release seriously.
> Fortunately, it doesn't look like he ever tried it.
I tried, but I'm completely paralyzed here. As long as I can't see a straightforward way to integrate support into the typechecker, I won't write a single line.
Also, why "fortunately"?
> as if the developer did think of these as opposites
Not opposites, just not aligned with each other. The demands of Scheme's runtime model inherently conflict with realtime performance. There's nothing wrong with that, but that sets it apart from C.
With this sentence, I hoped to address Scheme users who wish they could generate faster, tightly-constrained assembly while not having to give up on hygienic macros and nested evaluation contexts. No insult was intended.
> Also, why "fortunately"?
Because I think it's a lot harder (both for you and for your users) than you thought at the time, and I hoped you wouldn't waste your time (and your users' time) on it. Personally, I think GC is the way to go to keep your (programmers') sanity. If you worry about GC pauses in time-critical sections, provide for a way to turn off the GC temporarily for such sections.
You might want to mark such non-GC sections/functions specifically and enforce statically that no dynamic allocation may be triggered from within them. Kind of like a monad. No allocations, no GCs, no headaches. Simpler and more flexible than the Rust model.
> No insult was intended.
I didn't read that as such!
> Because I think it's a lot harder (both for you and for your users) than you thought at the time, and I hoped you wouldn't waste your time (and your users' time) on it.
I believe it is possible to offer such a feature without declarative baggage. If it's not, then I don't want it ;-)
> Personally, I think GC is the way to go to keep your (programmers') sanity. If you worry about GC pauses in time-critical sections, provide for a way to turn off the GC temporarily for such sections.
It might be even possible to leverage Scopes' support for compile-time introspection to write contextual GC's from within the language. I haven't explored that yet. In games development we usually prefer to allocate in frames, stages or on custom stacks, so that's also a way to do it.
Installing a GC at the lowest level is a brutal choice that precludes many other possible ways this could go, so I'm cautious.
FWIW, I’m in the camp of people who find this to be extremely interesting, but only given that there’s no GC.
S-expressions are not a special case of the syntax, S-expressions are the syntax. There are two notations, one is naked, python-looking, and one is traditionally braced, lispy-looking. It is possible to enter from one notation into the other at any point, depending on what suits the situation best.
Scopes has no garbage collection. There will be garbage collection for compile time symbols at some point, but the runtime is completely manually managed. That is a big difference to Scheme, although Scheme has been a major influence to Scopes design, and features that I believe to set Lisp/Scheme apart from other languages have been adopted.
Scopes is statically typed, but type signatures are not elementary to function definitions and value declarations, as with Scheme. It is a little limited with its compile-time closures though, whose application as first class values is limited.
interesting: supports a LLVM as well as a SPIR-V backend
"Snopes". Read this as "Snopes."
dyslexia won't give you any scopes!