If "Hyperfiddle is after more of a vision where like, what would it take for my mom or my future kids to be programmers and yet not be professional software engineers." then I suggest you focus on drastically simplifying the vocabulary you use to explain what Hyperfiddle is.
For example:
"This is a pastebin like environment for learning Hyperfiddle.
Please namespace :idents with your username
Schema changes are allowed
To make a fiddle interactive, change tank.hyperfiddle.net to demo.hyperfiddle.net"
My mom no grok: pastebin, :idents, namespace as a verb, schema, "change" without a where to change.
Granted, this is a conversation between to engineers, so there's lexicon, but even listening, I get the sense that words get used without definition, or at least without a definition that speaker and listener share.
For those out of the loop - At Hyperfiddle we are building a tool for Clojure/Datomic programmers to make back of house dashboards easily. http://www.hyperfiddle.net/ Because Hyperfiddle/Datomic has such a simple information model, we see a path to eventually making this easy enough for code-savvy amateurs. But today you need to know Clojure.
How did you uncover the pain point around making these internal dashboards? What kind of resistance have you encountered from developers who typically enjoy rolling things from scratch?
Mostly the resistance is skepticism that this is even possible, which is totally justified because we weren't even sure ourselves until sometime last year. We ourselves are skeptical of all the Javascript and SQL centric solutions, including GraphQL. We see Clojure/Datomic as an opportunity to pave over all that complicated baggage and re-present programming in an accessible way. For example, a code-savvy product owner or data manager can make their own tools or diagnose problems without going through engineering, because the system is so operationally transparent.
Clojure programmers seem to be excited once they "get it". Hyperfiddle is hyper-extensible and built up in layers, most things can be customized with a Clojure/Script function. For example React.js views and Datomic transactions – in Hyperfiddle these are very simple, very often one-liners.
For some context on "what Niko presented at the Conj",
https://2018.clojure-conj.org/nikolas-gobel/
which is a reactive Datalog implementation built over differential dataflow. The .. um reported anxiety that to add new queries requires a recompile loop, replaying all of history, etc, suggests maybe they aren't entirely on top of how the system works.
hi Frank, we are definitely not entirely on top of this stuff, but it is very awesome and we will get there eventually :)
Can you help me understand where we went awry here:
"Now of course the problem there with a rules engine... I have to look at the incremental dataflow, but I imagine it's kind of the same sort of approach. The problem with that is that if you want to sort by a new thing or you have a new piece of information, you have to change the network and backflow any... [You have to know compile time...] Yeah. Yeah. Exactly. And there are systems where you can add a new rule at runtime, but a lot of those also involve then re-ingesting history into the network to repopulate anything they... 'Cause a lot of these systems to save memory, will just throw away any information you're not interested in, from compiled query."
If we are close can you tighten up the language and if we are far where did we go wrong? (In the context of differential dataflow, not rules networks)
Niko's talk should probably be the reference material, but you definitely can deploy new queries interactively, against maintained indexed representations of the historical streams. No recompilation required. Streams are maintained in indexed form and don't need to be re-flowed (they may need to be re-indexed if your query calls for an index that doesn't exist).
Deploying a new query takes ~milliseconds if the correct indices exist, and if not it seems to be no slower than purpose-built Datalog systems (e.g. LogicBlox, but performance seems solid compared with Souffle). This is despite then running against live-updated data.
As I suspect you know, we cannot report performance relative to Datomic. ;)
If "Hyperfiddle is after more of a vision where like, what would it take for my mom or my future kids to be programmers and yet not be professional software engineers." then I suggest you focus on drastically simplifying the vocabulary you use to explain what Hyperfiddle is.
For example: "This is a pastebin like environment for learning Hyperfiddle. Please namespace :idents with your username Schema changes are allowed To make a fiddle interactive, change tank.hyperfiddle.net to demo.hyperfiddle.net"
My mom no grok: pastebin, :idents, namespace as a verb, schema, "change" without a where to change.
Granted, this is a conversation between to engineers, so there's lexicon, but even listening, I get the sense that words get used without definition, or at least without a definition that speaker and listener share.
For those out of the loop - At Hyperfiddle we are building a tool for Clojure/Datomic programmers to make back of house dashboards easily. http://www.hyperfiddle.net/ Because Hyperfiddle/Datomic has such a simple information model, we see a path to eventually making this easy enough for code-savvy amateurs. But today you need to know Clojure.
How did you uncover the pain point around making these internal dashboards? What kind of resistance have you encountered from developers who typically enjoy rolling things from scratch?
Mostly the resistance is skepticism that this is even possible, which is totally justified because we weren't even sure ourselves until sometime last year. We ourselves are skeptical of all the Javascript and SQL centric solutions, including GraphQL. We see Clojure/Datomic as an opportunity to pave over all that complicated baggage and re-present programming in an accessible way. For example, a code-savvy product owner or data manager can make their own tools or diagnose problems without going through engineering, because the system is so operationally transparent.
Clojure programmers seem to be excited once they "get it". Hyperfiddle is hyper-extensible and built up in layers, most things can be customized with a Clojure/Script function. For example React.js views and Datomic transactions – in Hyperfiddle these are very simple, very often one-liners.
For some context on "what Niko presented at the Conj",
which is a reactive Datalog implementation built over differential dataflow. The .. um reported anxiety that to add new queries requires a recompile loop, replaying all of history, etc, suggests maybe they aren't entirely on top of how the system works.Edit: and again tomorrow at clojureD:
hi Frank, we are definitely not entirely on top of this stuff, but it is very awesome and we will get there eventually :)
Can you help me understand where we went awry here:
"Now of course the problem there with a rules engine... I have to look at the incremental dataflow, but I imagine it's kind of the same sort of approach. The problem with that is that if you want to sort by a new thing or you have a new piece of information, you have to change the network and backflow any... [You have to know compile time...] Yeah. Yeah. Exactly. And there are systems where you can add a new rule at runtime, but a lot of those also involve then re-ingesting history into the network to repopulate anything they... 'Cause a lot of these systems to save memory, will just throw away any information you're not interested in, from compiled query."
If we are close can you tighten up the language and if we are far where did we go wrong? (In the context of differential dataflow, not rules networks)
No worries!
Niko's talk should probably be the reference material, but you definitely can deploy new queries interactively, against maintained indexed representations of the historical streams. No recompilation required. Streams are maintained in indexed form and don't need to be re-flowed (they may need to be re-indexed if your query calls for an index that doesn't exist).
Deploying a new query takes ~milliseconds if the correct indices exist, and if not it seems to be no slower than purpose-built Datalog systems (e.g. LogicBlox, but performance seems solid compared with Souffle). This is despite then running against live-updated data.
As I suspect you know, we cannot report performance relative to Datomic. ;)
also see the healthy reddit clojure discussion: https://www.reddit.com/r/Clojure/comments/as16f6/transcript_...
I love Hyperfiddle, but I'm not good enough to use it.
Hyperfiddle 2.0 landing soon!
\o/ Can I squeeze a last minute request? Allow to login to http://demo.hyperfiddle.net for users who don't have a Google account.
In any case, I love the idea. Looking forward!