Experimental Superset of Node to support JavaScript Types in functions.

[u/EveYogaTech]

https://github.com/flowagi-eu/jstf

[u/frogic]

Uh. I think you might have accidentally created the most performance nightmare thing I've ever seen.

[u/EveYogaTech → u/frogic]

Probably. It's not for everyone or to put types literally everywhere.

However there are still many places where you want to quickly check for types at runtime.

The function op (as a single NodeJS function execution) gets about 5x slower, yet this is quite insignificant compared to the vast amount of operations needed for real work like handling a HTTP request.

[u/frogic → u/EveYogaTech]

Why would you want to auto generate the run time type checking though? Zod exists for when you do want to do that on complex objects but its reasonably easy to roll your own for specific type checks. Transpiling a code base to check the types of every single declared variable makes no sense. Think about how complex this gets too.

I have interface SomeComplexObject {thing: string; otherThing: GUID; anotherThing: SuperComplexTypeThatHasSomeComplexObjectsInThem}

So now we have a function that takes a couple objects that could have SomeComplexObject in it. Now consider that you might call that function in a loop. Or even just reduce an array that has some amount of them in it. Now we often type reduce now in typescript. This is just off the top of my head and isn't that contrived and we're adding a tonne of worthless code here that could easily be adding many many millions of useless operations. Typed languages don't just type check everything at run time they do it at compile time like typescript does. If your types are wrong at run time you either are lying to the compiler(which can be done with any language) or trusting data from another source. Things like Pedantic in python, Zod in JS and dotnet exist to fix the 'we got types and they might not be real' solve those.

[u/EveYogaTech → u/frogic]

I don't want to use Zod and all it's messy functions.

I simply want to quickly define JavaScript primitives and classes in my function arguments and validate them at runtime.

[u/frogic → u/EveYogaTech]

Okay how about we go about this another way. Explain to me your use case for type checking exhaustively all functions at run time.

[u/mmomtchev → u/frogic]

This debate is as old as computer languages are.

Additional runtime checks definitely help to spot bugs earlier, this has been tried and verified over the years. However, when left to their own devices, people generally tend to favour more flexible languages which are less hassle.

[u/frogic → u/mmomtchev]

I'm not arguing against run time validation. I'm arguing that transpiling every function to type check every parameter at invocation is madness. Nothing exists that does this. There is no benefit to exhautively checking every function parameter ever every time any function is run. A function can be called a million times. A function can have many functions in it. I guess the upside is that the Javascript runtime will likely scrub out the checks as unneccessary anyway.

[u/EveYogaTech → u/frogic]

> I'm arguing that transpiling every function to type check every parameter at invocation is madness.

I sort of agree with that as well, in principle, but in practice it's actually quite effective and the overhead of let's say a real API call is very low.

In the most optimal scenario we'd arrive at a JS runtime that doesn't just add additional checks through transpilation, but natively enforces the types.

[u/EveYogaTech → u/frogic]

Zod:

```js import { z } from "zod"; const userIdSchema = z.number();

function getUserById(userId) { userIdSchema.parse(userId);

return db.users.find(user => user.id === userId); } ```

jstf:

js function getUserById(userId: number) { return db.users.find(user => user.id === userId); }

[u/frogic → u/EveYogaTech]

That's not a use case that's just an example of code. In a real world scenario you would never use zod for that. You would just do a simple typeof but you wouldn't do that there because you aren't going to be calling getUserById with a variable that you don't know is a number. If userId comes from a network call you parse it on the edge with any number of libraries and frameworks. If its a local variable you don't typecheck it.

The only thing I can perceive you might be trying to solve is someone lying to typescript but you can lint and configure your way out of that.

[u/EveYogaTech → u/frogic]

That's not true, for example you could configure these functions to be directly called via HTTP input as a registry.

As I said before in the full reddit thread, there are many places where you want to quickly add type validation.

Other use-cases might by at other critical points like a TCP server, or any other place where you're handling user input.

[u/EveYogaTech → u/EveYogaTech]

Script used for benchmarking:

``` function add(a: number, b: number) { return a + b; }

const iterations = 10_000_00000;

console.time("runtime");

let result = 0;

for (let i = 0; i < iterations; i++) { result += add(i, 1); }

console.timeEnd("runtime"); console.log(result); ```

[u/alex-weej]

Obligatory: https://github.com/akutruff/typescript-needs-types

[u/EveYogaTech → u/alex-weej]

Nice collection, however it assumes that we cannot find a way to support types natively and faster in the future of Node/similar JS runtime.

Bun for example, also just transpiles TypeScript.

[u/fucking_passwords → u/EveYogaTech]

I feel like this would be better as a decorator / annotation for TS that builds in the runtime checks.. something like:

@RuntimeTypeChecks

function foo(bar: number, baz: string) {}

That way you could avoid runtime type checks for every single function and just focus on the ones that process unknown input. I get that you are positing that TS is too verbose/complex but we will have to agree to disagree on that one

[u/EveYogaTech → u/fucking_passwords]

This could work if your goal is to extend TypeScript.

The goal of this project is to extend JavaScript.

[u/EveYogaTech → u/EveYogaTech]

I'd also add that decorators might be tricky, because you (or a junior engineer) might forget the decorator and on a quick look it looks like types are defined while in reality they are only transpiled.

[u/its_jsec]

https://github.com/flowagi-eu/jstf/blob/main/bin/jstf.js

What in the world did I just read.

[u/gdmr458 → u/its_jsec]

holy shit, i was too optimist, before clicking the link op posted i thought they cloned https://github.com/nodejs/node and wrote a lot of C++ code to make this work

[u/EveYogaTech → u/gdmr458]

Not yet, but it might be needed in the long-term.

[u/EveYogaTech → u/its_jsec]

It works though.

[u/its_jsec → u/EveYogaTech]

Does it?

Your findFunctions regex only checks for `function {more matching}`. How does it handle functions as const expressions?

The node_modules check uses “/node_modules/“, which won’t match on Windows.

And this isn’t a superset of Node (how one would superset a runtime is beyond me).

This looks like you’re running a .js file through esbuild’s TypeScript loader, then splicing checks into the emitted output based on a captured function name (with the name map being a flat map, so a function that has the same name as a closure-scoped function will yield interesting results). Why not use an AST tree?

[u/boneskull → u/its_jsec]

parsing the AST would be even slower!

[u/its_jsec → u/boneskull]

Oh I know, but it might actually help accomplish what OP is attempting to do.

[u/boneskull → u/its_jsec]

don’t encourage them 😜

[u/EveYogaTech → u/its_jsec]

Thanks for the feedback!

[u/boneskull]

Apologies OP, but this is truly cursed. I think you will find it is better in the long run to actually write these type guards. You don’t need to use Zod or whatever, but the current implementation is not at all robust and has quite a few problems beyond simply performance.

You mention this is for an open-source project; if you have hope of gaining contributors you may want to steer away from something like this.

[u/EveYogaTech → u/boneskull]

I agree the current implementation is not yet super robust and experimental.

However I'd say it depends on what's possible and feasible in the long-term.

We could even add meta data about types to functions (e.g. directly in v8/webkit) and in essence "compile JavaScript".

[u/Xacius]

use runtime libraries like Zod

• https://arktype.io/
• https://zod.dev/

/thread

[u/lepepls → u/Xacius]

you can't /thread your own comment
learn2internet

[u/Beginning-Seat5221]

All I'm going to say is that Node.js isn't a language, it's a runtime. I'm not sure if you can have a superset of Node.js.

I think "superset of JavaScript" is probably what you're aiming at here?

You'd also probably be better off making a TypeScript pre-compiler or post-compiler - keep the surface language TS for TS tooling/ecosystem and just use comments to opt functions in for a pre-compile or post-compile that keeps each stage valid TS or JS compatible with everything in the ecosystem (although people have been doing this with babel for years already).

[u/DLabz]

Javascript + JSDOC + //@ts-check is what works for me.

You can go wild with defining parameters and IDE will do type check your code, show hints and snippets when you hover a function, show documentatation, generate AST graphs, link debugger breakpoints back and what not, all without ever needing a compilation step to run or enforcing types at runtime, which saves you from needing to explicitly dode every and any special/edge/cornet case for your code to run, while allowing you to transpile it to TypeScript using tsc.

At the end of the day, loosely typed is a feature, not a deficit.

[u/EveYogaTech → u/DLabz]

That can appear to work, but it still doesn't check types at runtime (e.g. when you are working with dynamic incoming user/API data).

[u/DLabz → u/EveYogaTech]

That’s a good thing, because I can pass just about anything to a `(o)=>o.k + o.v;` function and it will not break.

When I want a tighter check I’d use decomposition `({k,v})=>k+v` set defaults, or check specifically ([m=2,n=3,…r]) => [m, n, … r.filter((n)=>! isNan(n)]

I don’t need to write a separate dotVector for 2D, 3D or nD

[u/Positive_Method3022]

Bro! 😅

[u/EveYogaTech]

A bit of background:

I tried both JavaScript (Node) and TypeScript for our open-source solution, but kept running into the problem of actually needing to validate types at many places, primarily in functions.

Initially TypeScript seems to be the solution, but if you look at the compiled source code you will see that there is actually no type checking at all, after you compiled your TypeScript to JavaScript, which at runtime can cause problems.

The common solution for this is to then use runtime libraries like Zod, practically often only at the HTTP layer.

JSTF takes a different approach where you can (optionally) define types in all your functions similar to TypeScript that are checked at runtime.

[u/CreamyJala → u/EveYogaTech]

> You look at the compiled source code you will see that there is no type checking at all

That’s kind of the point of Typescript, though. You’re able to catch the obvious problems ahead of time, get out nearly identical JavaScript code from the compiler, and still be able to debug with ease.

If your goal is safety and avoiding problems, then you would’ve considered the implications of what could (and will) go wrong when injecting “invisible” code like this, especially with projects that have multiple collaborators.

If your goal was to have fun and not use this seriously, then go for it - have fun with it! If you’re convinced that this is the proper approach and are going to use it on anything important, just be aware of the risks and footguns, and maybe chill on your tokens for a bit

[u/EveYogaTech → u/CreamyJala]

> considered the implications of what could (and will) go wrong when injecting “invisible” code

This is broadly true for every compiler ever made. In practice, I don't think many people besides reverse engineers are debugging machine code.

[u/boneskull → u/EveYogaTech]

that’s not what the commenter meant. you need source maps or debugging will be excruciating

[u/EveYogaTech → u/boneskull]

A source map is indeed a pretty good idea.