uPlot maintainer here. this looks interesting, i'll do a deeper dive soon :)
some notes from a very brief look at the 1M demo:
- sampling has a risk of eliminating important peaks, uPlot does not do it, so for apples-to-apples perf comparison you have to turn that off. see https://github.com/leeoniya/uPlot/pull/1025 for more details on the drawbacks of LTTB
- when doing nothing / idle, there is significant cpu being used, while canvas-based solutions will use zero cpu when the chart is not actively being updated (with new data or scale limits). i think this can probably be resolved in the WebGPU case with some additional code that pauses the updates.
- creating multiple charts on the same page with GL (e.g. dashboard) has historically been limited by the fact that Chrome is capped at 16 active GL contexts that can be acquired simultaneously. Plotly finally worked around this by using https://github.com/greggman/virtual-webgl
> data: [[0, 1], [1, 3], [2, 2]]
this data format, unfortunately, necessitates the allocation of millions of tiny arrays. i would suggest switching to a columnar data layout.
Really appreciate you taking the time to look, Leon - uPlot has been a huge inspiration for proving that browser charts don't have to be slow.
Both points are fair:
1. LTTB peak elimination - you're right, and that PR is a great reference. For the 1M demo specifically, sampling is on by default to show the "it doesn't choke" story. Users can set sampling: 'none' for apples-to-apples comparison. I should probably add a toggle in the demo UI to make that clearer.
2. Idle CPU - good catch. Right now the render loop is probably ticking even when static. That's fixable - should be straightforward to only render on data change or interaction. Will look into it.
Would love your deeper dive feedback when you get to it. Always more to learn from someone who's thought about this problem as much as you have.
I once had to deal with many million data points for an application. I ended up mip-mapping them client-side.
But regarding sampling, if it's a line chart, you can sample adaptively by checking whether the next point makes a meaningfully visible difference measured in pixels compared to its neighbours. When you tune it correctly, you can drop most points without the difference being noticeable.
I didn't find any else doing that at the time, and some people seemed to have trouble accepting it as a viable solution, but if you think about it, it doesn't actually make sense to plot say 1 million points in a line chart 1000 pixels wide. On average that would make 1000 points per pixel.
Is there any techniques using wavelet decomposition to decimate the high frequency component while retaining peaks? I feel like that's a more principled approach than sampling but I haven't seen any literature on it describing the specific techniques (unless the idea is fundamentally unsound which is not obvious to me).
Interesting idea - I haven't explored wavelet-based approaches but the intuition makes sense: decompose into frequency bands, keep the low-frequency trend, and selectively preserve high-frequency peaks that exceed some threshold.
My concern would be computational cost for real-time/streaming use cases. LTTB is O(n) and pretty cache-friendly. Wavelet transforms are more expensive, though maybe a GPU compute shader could make it viable.
The other question is whether it's "visually correct" for charting specifically. LTTB optimizes for preserving the visual shape of the line at a given resolution. Wavelet decomposition optimizes for signal reconstruction - not quite the same goal.
That said, I'd be curious to experiment. Do you have any papers or implementations in mind? Would make for an interesting alternative sampling mode.
I don't. I just remember watching a presentation on it and it always struck me that wavelets are an incredibly powerful and underutilized technique for data reduction while preserving quality in a quantifiable and mathematically justifiable way.
I don't have any papers in mind, but I do think that the critique around visual shape vs signal reconstruction may not be accurate given that wavelets are starting to see a lot of adoption in the visual space (at least JPEG2000 is the leading edge in that field). Might also be interesting to use DCT as well. I think these will perform better than LTTB (of course the compute cost is higher but there's also HW acceleration for some of these or will be over time).
What I did in a few projects to plot aggregated (resampled) data without loosing peaks was to plot it over an area chart representing the min-max values before aggregating (resampling). It worked pretty well.
Update: Patched idle CPU usage while nothing is being rendered.
One thing to note: I added a toggle to "Benchmark mode" in the 1M benchmark example - this preserves the benchmark capability while demonstrating efficient idle behavior.
Another thing to note: Do not be alarmed when you see the FPS counter display 0 (lol), that is by design :) Frames are rendered efficiently. If there's nothing to render (no dirty frames) nothing is rendered. The chart will still render at full speed when needed, it just doesn't waste cycles rendering the same static image 60 times per second.
Blown away by all of you amazing people and your support today :)
If you have tons of datapoints, one cool trick is to do intensity modulation of the graph instead of simple "binary" display. Basically for each pixel you'd count how many datapoints it covers and map that value to color/brightness of that pixel. That way you can visually make out much more detail about the data.
In electronics world this is what "digital phosphor" etc does in oscilloscopes, which started out as just emulating analog scopes. Some examples are visible here https://www.hit.bme.hu/~papay/edu/DSOdisp/gradient.htm
Great suggestion - density mapping is a really effective technique for overplotted data. Instead of drawing 1M points where most overlap, you're essentially rendering a heatmap of point concentration. WebGPU compute shaders would be perfect for this - bin the points into a grid, count per cell, then render intensity. Could even do it in a single pass. I've been thinking about this for scatter plots especially, where you might have clusters that just look like solid blobs at full zoom-out. A density mode would reveal the structure. Added to the ideas list - thanks for the suggestion!
Drawing lots of single pixels with alpha blending is probably one of the least efficient ways to use the rasterizer though. A good compute shader implementation would be substantially faster.
At 1M points it hardly makes a difference. Besides, 1 point -> 1 pixel mapping is good enough for a demo, but in practice it will produce nasty aliasing artifacts because real datasets aren't aligned with pixel coordinates. So you have to draw each point as a 2x2 square at least with precise shading, and we are back to the rasterizer pipeline. Edit: what actually needs to be computed is the integral of the points dataset over each square pixel, and that depends on the shape of each point, even if it's smaller than a pixel.
Aren't we at petaflops now with GPUs? 1M or even 1G points should be no issue if it renders to a framebuffer and doesn't go through mountains af JS framework rubbish followed by mountains of GTK/Qt/.NET rubbish.
That works if more overdraw = more intensity is all you care about, and may very well be good enough for many kinds of charts. But with heat map plots one usually wants a proper mapping of some intensity domain to a color map and a legend with a color gradient that tells you which color represents which value. Which requires binning, counting per bin, and determining the min and max values.
agreed, heatmaps with logarithmic cell intensity are the way to go for massive datasets in things like 10,000-series line charts and scatter plots. you can generally drill downward from these, as needed.
We’ve been working on a browser-based Link Graph (osint) analysis tool for months now (https://webvetted.com/workbench). The graph charting tools on the market are pretty basic for the kind of charting we are looking to do (think 1000s of connected/disconnected nodes/edges. Being able to handle 1M points is a dream.
That's a cool project! Just checked out the workbench. I should be upfront though: ChartGPU is currently focused on traditional 2D charts (line, bar, scatter, candlestick, etc.), not graph/network visualization with nodes and edges. That said, the WebGPU rendering patterns would translate well to force-directed graphs. The scatter renderer already handles thousands of instanced points - extending that to edges wouldn't be a huge leap architecturally.
Is graph visualization something you'd want as part of ChartGPU, or would a separate "GraphGPU" type library make more sense? Curious how you're thinking about it.
Really fantastic work! Can't wait to play around with your library. I did a lot of work on this at a past job long ago and the state of JS tooling was so inadequate at the time we ended up building an in-house Scala visualization library to pre-render charts...
More directly relevant, I haven't looked at the D3 internals for a decade, but I wonder if it might be tractable to use your library as a GPU rendering engine. I guess the big question for the future of your project is whether you want to focus on the performance side of certain primitives or expand the library to encompass all the various types of charts/customization that users might want. Probably that would just be a different project entirely/a nightmare, but if feasible even for a subset of D3 you would get infinitely customizable charts "for free." https://github.com/d3/d3-shape might be a place to look.
In my past life, the most tedious aspect of building such a tool was how different graph standards and expectations are across different communities (data science, finance, economics, natural sciences, etc). Don't get me started about finance's love for double y-axis charts... You're probably familiar with it, but https://www.amazon.com/Grammar-Graphics-Statistics-Computing... is fantastic if you continue on your own path chart-wise and you're looking for inspiration.
Thanks - and great question about direction. My current thinking: Focus on performance-first primitives for the core library. The goal is "make fast charts easy" not "make every chart possible." There are already great libraries for infinite customization (D3, Observable Plot) - but they struggle at scale.
That said, the ECharts-style declarative API is intentionally designed to be "batteries included" for common cases. So it's a balance: the primitives are fast, but you get sensible defaults for the 80% use case without configuring everything. Double y-axis is a great example - that's on the roadmap because it's so common in finance and IoT dashboards. Same with annotations, reference lines, etc. Haven't read the Grammar of Graphics book but it's been on my list - I'll bump it up. And d3-shape is a great reference for the path generation patterns. Thanks for the pointers!
Question: What chart types or customization would be most valuable for your use cases?
Most of my use cases these days are for hobby projects, which I would bucket into the "data science"/"data journalism" category. I think this is the easiest audience to develop for, since people usually don't have any strict disciplinary norms apart from clean and sensible design. I mention double y-axes because in my own past library I stupidly assumed no sensible person would want such a chart -- only to have to rearchitect my rendering engine once I learned it was one of the most popular charts in finance.
That is, you're definitely developing the tool in a direction that I and I think most Hacker News readers will appreciate and it sounds like you're already thinking about some of the most common "extravagances" (annotations, reference lines, double y-axis etc). As OP mentioned, I think there's a big need for more performant client-side graph visualization libraries, but that's really a different project. Last I looked, you're still essentially stuck with graphviz prerendering for large enough graphs...
Ha - the double y-axis story is exactly why I want to get it right. Better to build it in properly than bolt it on later.
"Data science/data journalism" is a great way to frame the target audience. Clean defaults, sensible design, fast enough that the tool disappears and you just see the data.
And yeah, graphviz keeps coming up in this thread - clearly a gap in the ecosystem. Might be a future project, but want to nail the 2D charting story first and foremost.
Thanks for the thoughtful feedback - this is exactly the kind of input that shapes the roadmap.
my 2 cents: I'm one of these people that could possibly use your tool. However, the website doesnt give me much info. I'd urge you to add some more pages that showcase the product and what it can do with more detail. Would help capture more people imo.
TimeLine maintainer here. Their demo for live-streamed data [0] in a line plot is surprisingly bad given how slick the rest of it seems. For comparison, this [1] is a comparatively smooth demo of the same goal, but running entirely on the main thread and using the classic "2d" canvas rendering mode.
That was obvious before even looking at the repo because the OP used "the core insight" in the intro. Other telltale signs of these type of AI projects:
- new account
- spamming the project to HN, reddit etc the moment the demo half works
- single contributor repo
- Huge commits minutes apart
- repo is less than a week old (sometimes literally hours)
- half the commits start with "Enhance"
- flashly demo that hides issues immediately obvious to experts in the field
- author has slop AI project(s)
OP uses more than one branch so he's more sophisticated than most.
What's the best way to get all those points from a backend into the frontend webgpu compute shader?
There doesn't seem to be a communication mechanism that has minimal memcopy or no serialization/deserialization, the security boundary makes this difficult.
I have a backend array of 10M i16 points, I want to get this into the frontend (with scale & offset data provided via side channel to the compute shader).
As it stands, I currently process on the backend and send the frontend a bitmap or simplified SVG. I'm curious to know about the opposite approach.
Not sure, but I solved a similar problem many years ago, and ended up concluding it was silly to send all the data to the client when the client didn't have the visual resolution to show it anyway. So I sampled it adaptively client-side by precomputing and storing multiple zoom-levels. That way the client-side chart app would get the points and you could zoom in, but you'd only ever retrieve about 1000-2000 points at the time.
@huntergemmer - assuming you are the author, curious about your experience using .claude and .cursor, I see sub agents defined under these folders, what percent of your time spent would you say is raw coding vs prompting working on this project? And perhaps any other insights you may have on using these tools to build a library - see your first commit was only 5 days ago.
> I kept hitting the same wall: charting libraries that claim to be "fast" but choke past 100K data points
Haha, Highcharts is a running joke around my office because of this. Every few years the business will bring in consultants to build some interface for us, and every time we will have to explain to them that highcharts, even with it's turbo mode enabled chokes on our data streams almost immediately.
I just rewrote all the graphs on phrasing [1] to webgl. Mostly because I wanted custom graphs that didn’t look like graphs, but also because I wanted to be able to animate several tens of thousands of metrics at a time.
After the initial setup and learning curve, it was actually very easy. All in all, way less complicated than all the performance hacks I had to do to get 0.01% of the data to render half as smooth using d3.
Although this looks next level. I make sure all the computation happens in a single o(n) loop but the main loop still takes place on the cpu. Very well done
To anyone on the fence, GPU charting seemed crazy to me beforehand (classic overengineering) but it ends up being much simpler (and much much much smoother) than traditional charts!
Wow, this is great. I practically gave up on rendering large data in EasyAnalytica because plotting millions of points becomes a bad experience, especially in dashboards with multiple charts. My current solution is to downsample to give an “overview” and use zoom to allow viewing “detailed” data, but that code is fragile.
One more issue is that some browser and OS combinations do not support WebGPU, so we will still have to rely on existing libraries in addition to this, but it feels promising.
Can it scroll while populating? I was trying to heart rate chart using libs which is captured at 60fps from camera (finger on camera with flash light). Raw drawing with canvas was faster than any libs.
Drawing and scrolling live data was problem for a lib (dont remember which one) because it was drawing the whole thing on every frame.
I've always been a bit skeptical of JS charting libs that want to bring the entire data to the client and do the rendering there, vs at least having the option to render image tiles on the server and then stream back tooltips and other interactive elements interactively.
However, this is pretty great; there really aren't that many use cases that require more than a million points. You might finally unseat dygraphs as the gold standard in this space.
> render image tiles on the server and then stream back tooltips and other interactive elements interactively.
I guess the real draw here is smooth scrolling and zooming, which is hard to do with server-rendered tiles. There's also the case of fully local use, where server rendering doesn't make much sense.
> I've always been a bit skeptical of JS charting libs that want to bring the entire data to the client and do the rendering there
The computer on my desk only costs me the electric power to run it, and there's 0 network latency between it and the monitor on which I'm viewing charts. If I am visualizing some data and I want to rapidly iterate on the visualization or interact with it, there's no more ideal place for the data to reside than right there. DDR5 and GPUs will be cheap again, some day.
> I've always been a bit skeptical of JS charting libs that want to bring the entire data to the client and do the rendering there, vs at least having the option to render image tiles on the server and then stream back tooltips and other interactive elements interactively.
I agree, unfortunately no library I've found supports this. I currently SSR plots to SVG using observable plot and JSDom [0]. This means there is no javascript bundle, but also no interactivity, and observable doesn't have a method to generate a small JS sidecar to add interactivity. I suppose you could progressive enhance, but plot is dozens of kilobytes that I'd frankly rather not send.
There's no question that it's a huge step up in complexity to wire together such tightly-linked front and backend components, but it is done for things like GIS, where you want data overlays.
I think it's just a different mindset; GIS libs like Leaflet kind of assume they're the centerpiece of the app and can dictate a bunch of structure around how things are going to work, whereas charting libs benefit a lot more from "just add me to your webpack bundle and call one function with an array and a div ID, I promise not to cause a bunch of integration pain!"
Last time I tried to use it for dashboarding, I found Kibana did extremely aggressive down-sampling to the point that it was averaging out the actual extremes in the data that I needed to see.
I’ve had a lot of success rendering svg charts via Airbnb’s visx on top of React Server Components, then sprinkling in interactivity with client components. Worth looking into if you want that balance.
It’s more low level than a full charting library, but most of it can run natively on the server with zero config.
I’ve always found performance to be kind of a drag with server side dom implementations.
Cool to see that this project started 5 days ago!
Unfortunately, I can not make it work on my system (Ubuntu, chrome, WebGPU enabled as described in the documentation).
On the other hand, It works on my Android phone...
Funny enough, I am doing something very similar: a C++ portable (Windows, Linux MacOS) charting library, that also compile to WASM and runs in the browser...
I am still at day 2, so see you in 3 days, I guess!
Error message: "WebGPU Error: Failed to request WebGPU adapter. No compatible adapter found. This may occur if no GPU is available or WebGPU is disabled.".
The rendering is very cool, but what i really want is this as a renderer i can plug into Vega.
Vega/VGlite have amazing charting expressivity in their spec language, most other charting libs don't come close. It would be very cool to be able to take advantage of that.
Please support a fallback, ideally a 2D one too. WebGPU and WebGL are a privacy nightmare and the former is also highly experimental. I don't mind sub-60 FPS rendering, but I'd hate having to enable either of them just to see charts if websites were to adopt this library.
The web is already bad requiring JavaScript to merely render text and images. Let's not make it any worse.
And even if WebGPU is enabled, the implementation might still be broken or inefficient in various ways. For example, Firefox uses some ridiculous polling-based approach [1] to check for completion, which disqualifies the implementation for many performance-critical applications.
Apparently you can turn it on with about:config / dom.webgpu.enabled
But personally, I'm not going to start turning on unsafe things in my browser so I can see the demo. I tried firefox and chromium and neither worked so pfft, whatever.
In Chromium it's been good for a good while, judges still out on when it'll be good in Firefox. Safari I have no clue about, nor whatever Microsoft calls their browser today.
Fun benchmark :) I'm getting 165 fps (screen refresh rate), 4.5-5.0 in GPU time and 1.0 - 1.2 in CPU time on a 9970x + RTX Pro 6000. Definitely the smoothest graph viewer I've used in a browser with that amount of data, nicely done!
Would be great if you had a button there one can press, and it does a 10-15 second benchmark then print a min/max report, maybe could even include loading/unloading the data in there too, so we get some ranges that are easier to share, and can compare easier between machines :)
165 fps on that setup - that's awesome to hear! Thanks for testing on high-end hardware.
Love the benchmark button idea. A "Run Benchmark" mode that captures:
- Load time
- GPU time
- CPU time
- Min/max/avg FPS over 10-15 seconds
- Hardware info
Then export a shareable summary or even a URL with encoded results. Would make for great comparison threads.
Adding this to the roadmap - would make a great v0.2 feature. Thanks for the suggestion!
I've been looking for a followup to uPlot - Lee who made uPlot is a genius and that tool is so powerful, however I need OffscreenCanvas running charts 100% in worker threads. Can ChartGPU support this?
I started Opus 4.5 rewrite of uPlot to decouple it from DOM reliance, but your project is another level of genius.
I hope there is consideration for running your library 100% in a worker thread ( the data munging pre-chart is very heavy in our case )
Thanks! Leon's uPlot is fantastic - definitely an inspiration.
Worker thread support via OffscreenCanvas is a great idea and WebGPU does support it. I haven't tested ChartGPU in a worker context yet, but the architecture should be compatible - we don't rely on DOM for rendering, only for the HTML overlay elements (tooltips, axis labels, legend).
The main work would be:
1. Passing the OffscreenCanvas to the worker
2. Moving the tooltip/label rendering to message-passing or a separate DOM layer
For your use case with heavy data munging, you could also run just the data processing in a worker and pass the processed arrays to ChartGPU on the main thread - that might be a quicker win.
Would you open an issue on GitHub? I'd love to understand your specific workload better. This feels like a v0.2 feature worth prioritizing.
You have a good point about doing zero copy transferables which would probably work.
There is certainly something beautiful about your charging GPU code being part of a file that runs completely isolated in another thread along with our websocket Data fire hose
Architecturally that could be something interesting where you expose a typed API wrapping postmessage where consumers wanting to bind the main thread to a worker thread could provide the offscreen canvas as well as a stream of normalized, touch and pointer events, keyboard and wheel. Then in your worker listeners could handle these incoming events and treat them as if they were direct from the event listeners on the main thread; effectively, your library is thread agnostic.
I'd be happy to discuss this on GitHub. I'll try to get to that today. See you there.
I am on the same boat. Current user and a fan of uPlot starting to hit performance limits. Thank you for this library, I will start testing it soon.
On the topic of support for worker threads, in my current project I have multiple data sources, each handled by its own worker. Copying data between worker and main thread - even processed - can be an expensive operation. Avoiding it can further help with performance.
Very Nice. There is an issue with panning on the million point demo -- it currently does not redraw until the dragging velocity is below some threshold, but it should seem like the points are just panned into frame. It is probably enough to just get rid of the dragging velocity threshold, but sometimes helps to cache an entire frame around the visible range
Safari on latest Sequoia doesn't support this. Given that many people will not upgrade to the latest version, it is a shame Safari is behind these things.
What purposes have you found for rendering so many datapoints? It seems like at a certain point, say above a few thousand, it becomes difficult to discriminate/less useful to render more in many cases
this is so well done, thanks for sharing it. i've been trying to communicate with people how we are living in the golden age of dev where things that previously couldn't have been created, now can be. this is an amazing example of that.
Curious:
How does TradingView et.al. solves this problem? They should have the same limitations?
(actually, Im a user of the site, though I never started digging down how they made id)
Tradingview’s charts couldn’t handle a million data points. They typically just render a few thousand candlesticks at a time, which is trivial with well optimized Canvas code.
Thanks! Currently focused on 2D charts. That's where the "big data" performance problem is most painful.
3D is coming (it's the same rendering pipeline), but I'd want to get the 2D story solid first before expanding scope.
The slice animation is doable though - we already have animation infrastructure for transitions. An "explode slice on click" effect would be a fun addition to the pie/donut charts.
What's your use case? Dashboard visuals or something else?
No Firefox support? It has had WebGPU support since version 141.
Even when I turn on dom.webgpu.enabled, I still get "WebGPU is disabled by blocklist" even though your domain is not in the blocklist, and even if I turn on gfx.webgpu.ignore-blocklist.
I hope you have a way to monetize/productize this, because this has three.js potential. I love this. Keep goin! And make it safe (a way to fund, don't overextend via OSS). Good luck, bud.
Also, you are a master of naming. ChartGPU is a great name, lol!
Thanks! The name was honestly just "what does this do" + "how does it do it" haha.
Interesting you mention three.js - there's definitely overlap in the WebGPU graphics space. My focus is specifically on 2D data visualization (time series, financial charts, dashboards), but I could see the rendering patterns being useful elsewhere.
On sustainability - still figuring that out. For now it's a passion project, but I've thought about a "pro" tier for enterprise features (real-time collaboration, premium chart types) while keeping the core MIT forever. Open to ideas if you have thoughts.
Have you thought about leaning into some of the fintech space? They'd happily pay for the sorts of features they need to stream financial data (which is usually bazillions of data points) and graph it efficiently.
Off the top of my head, look into Order Book Heatmaps, 3D Volatility Surfaces, Footprint Charts/Volatility deltas. Integrating drawing tools like Fibonacci Retracements, Gann Fans etc. It would make it very attractive to people willing to pay.
The number of points actually being rendered doesn't seem to warrant the webgpu implementation. It's similar to the number of points that cubism.js could throw on the screen 15 years ago.
This is great, but I don't see it being useful for most use cases.
Most high-level charting libraries already support downsampling. Rendering data that is not visible is a waste of CPU cycles anyway. This type of optimization is very common in 3D game engines.
Also, modern CPUs can handle rendering of even complex 2D graphs quite well. The insanely complex frontend stacks and libraries, a gazillion ads and trackers, etc., are a much larger overhead than rendering some interactive charts in a canvas.
I can see GPU rendering being useful for applications where real-time updates are critical, and you're showing dozens of them on screen at once, in e.g. live trading. But then again, such applications won't rely on browsers and web tech anyway.
Soon there will be only 3 factors that we will care about: API (easy to use and integrate), behavior (does it do what I want it to do?) and testability (do I have sufficient guaranty that the code doesn't have errors).
The fact that the code was generated by a human or a machine is less and less important.
.c? what a yikes. I took a quick look at the code and this application doesn't even have any machine code in it, it's just words like "while", "for", "if", "else" and other English words - someone back in 1970s, I'm sure.
Yeah, hypers need to cool their language down a bit and llmuddites need to acquire a bit of nuance. New technology tends to create large camps initially on both sides :)
Because it is not reasonable to expend high effort to verify something that took no effort to create. That is not a workable long term solution. Instead you have to rely on low effort signals to signify if something is WORTH expending energy on.
Yeah, that'd be stupid right? I have the same mindset with projects and code. If it's a good project, it's a good project, I don't care what IDE they used. If the code is shit it's shit regardless of how it was produced. But at least point to something concrete so me and others could potentially learn something, or understand something better.
Agree, but, ah, can you illuminate. <totally-offtopic data-but="i am intesnely curious"> Quite amazing 6000 points in under 3 months. Yuge. OK a few almost 1K posts, but they drop off. You must have some mega-point comments. Can you, eh, "point me" to a comment of yours that has a super amount of upvotes?
Sorry if this is weird, it's just I've never personally experienced a comment with anything more than 100 - 200 points. And that was RARE. I totally get if you don't want to...but like, what were your "kilopoint" comments, or thereabouts? </offtopic>
I think if you too retire, have nothing specific to do for some months, and have too much free time to discuss with strangers on the internet about a wide range of topics you ideally have strong opinions about, you too can get more HN karma than you know what to do with :)
some notes from a very brief look at the 1M demo:
- sampling has a risk of eliminating important peaks, uPlot does not do it, so for apples-to-apples perf comparison you have to turn that off. see https://github.com/leeoniya/uPlot/pull/1025 for more details on the drawbacks of LTTB
- when doing nothing / idle, there is significant cpu being used, while canvas-based solutions will use zero cpu when the chart is not actively being updated (with new data or scale limits). i think this can probably be resolved in the WebGPU case with some additional code that pauses the updates.
- creating multiple charts on the same page with GL (e.g. dashboard) has historically been limited by the fact that Chrome is capped at 16 active GL contexts that can be acquired simultaneously. Plotly finally worked around this by using https://github.com/greggman/virtual-webgl
> data: [[0, 1], [1, 3], [2, 2]]
this data format, unfortunately, necessitates the allocation of millions of tiny arrays. i would suggest switching to a columnar data layout.
uPlot has a 2M datapoint demo here, if interested: https://leeoniya.github.io/uPlot/bench/uPlot-10M.html
Both points are fair:
1. LTTB peak elimination - you're right, and that PR is a great reference. For the 1M demo specifically, sampling is on by default to show the "it doesn't choke" story. Users can set sampling: 'none' for apples-to-apples comparison. I should probably add a toggle in the demo UI to make that clearer.
2. Idle CPU - good catch. Right now the render loop is probably ticking even when static. That's fixable - should be straightforward to only render on data change or interaction. Will look into it.
Would love your deeper dive feedback when you get to it. Always more to learn from someone who's thought about this problem as much as you have.
I once had to deal with many million data points for an application. I ended up mip-mapping them client-side.
But regarding sampling, if it's a line chart, you can sample adaptively by checking whether the next point makes a meaningfully visible difference measured in pixels compared to its neighbours. When you tune it correctly, you can drop most points without the difference being noticeable.
I didn't find any else doing that at the time, and some people seemed to have trouble accepting it as a viable solution, but if you think about it, it doesn't actually make sense to plot say 1 million points in a line chart 1000 pixels wide. On average that would make 1000 points per pixel.
My concern would be computational cost for real-time/streaming use cases. LTTB is O(n) and pretty cache-friendly. Wavelet transforms are more expensive, though maybe a GPU compute shader could make it viable.
The other question is whether it's "visually correct" for charting specifically. LTTB optimizes for preserving the visual shape of the line at a given resolution. Wavelet decomposition optimizes for signal reconstruction - not quite the same goal.
That said, I'd be curious to experiment. Do you have any papers or implementations in mind? Would make for an interesting alternative sampling mode.
I don't have any papers in mind, but I do think that the critique around visual shape vs signal reconstruction may not be accurate given that wavelets are starting to see a lot of adoption in the visual space (at least JPEG2000 is the leading edge in that field). Might also be interesting to use DCT as well. I think these will perform better than LTTB (of course the compute cost is higher but there's also HW acceleration for some of these or will be over time).
One thing to note: I added a toggle to "Benchmark mode" in the 1M benchmark example - this preserves the benchmark capability while demonstrating efficient idle behavior.
Another thing to note: Do not be alarmed when you see the FPS counter display 0 (lol), that is by design :) Frames are rendered efficiently. If there's nothing to render (no dirty frames) nothing is rendered. The chart will still render at full speed when needed, it just doesn't waste cycles rendering the same static image 60 times per second.
Blown away by all of you amazing people and your support today :)
In electronics world this is what "digital phosphor" etc does in oscilloscopes, which started out as just emulating analog scopes. Some examples are visible here https://www.hit.bme.hu/~papay/edu/DSOdisp/gradient.htm
We’ve been working on a browser-based Link Graph (osint) analysis tool for months now (https://webvetted.com/workbench). The graph charting tools on the market are pretty basic for the kind of charting we are looking to do (think 1000s of connected/disconnected nodes/edges. Being able to handle 1M points is a dream.
This will come in very handy.
Is graph visualization something you'd want as part of ChartGPU, or would a separate "GraphGPU" type library make more sense? Curious how you're thinking about it.
More directly relevant, I haven't looked at the D3 internals for a decade, but I wonder if it might be tractable to use your library as a GPU rendering engine. I guess the big question for the future of your project is whether you want to focus on the performance side of certain primitives or expand the library to encompass all the various types of charts/customization that users might want. Probably that would just be a different project entirely/a nightmare, but if feasible even for a subset of D3 you would get infinitely customizable charts "for free." https://github.com/d3/d3-shape might be a place to look.
In my past life, the most tedious aspect of building such a tool was how different graph standards and expectations are across different communities (data science, finance, economics, natural sciences, etc). Don't get me started about finance's love for double y-axis charts... You're probably familiar with it, but https://www.amazon.com/Grammar-Graphics-Statistics-Computing... is fantastic if you continue on your own path chart-wise and you're looking for inspiration.
That said, the ECharts-style declarative API is intentionally designed to be "batteries included" for common cases. So it's a balance: the primitives are fast, but you get sensible defaults for the 80% use case without configuring everything. Double y-axis is a great example - that's on the roadmap because it's so common in finance and IoT dashboards. Same with annotations, reference lines, etc. Haven't read the Grammar of Graphics book but it's been on my list - I'll bump it up. And d3-shape is a great reference for the path generation patterns. Thanks for the pointers!
Question: What chart types or customization would be most valuable for your use cases?
That is, you're definitely developing the tool in a direction that I and I think most Hacker News readers will appreciate and it sounds like you're already thinking about some of the most common "extravagances" (annotations, reference lines, double y-axis etc). As OP mentioned, I think there's a big need for more performant client-side graph visualization libraries, but that's really a different project. Last I looked, you're still essentially stuck with graphviz prerendering for large enough graphs...
"Data science/data journalism" is a great way to frame the target audience. Clean defaults, sensible design, fast enough that the tool disappears and you just see the data.
And yeah, graphviz keeps coming up in this thread - clearly a gap in the ecosystem. Might be a future project, but want to nail the 2D charting story first and foremost.
Thanks for the thoughtful feedback - this is exactly the kind of input that shapes the roadmap.
A lot of improvements are possible, based on 20 years of progress in interactive systems, and just overall computing performance.
https://chartgpu.github.io/ChartGPU/examples/million-points/...
While dragging, the slider does not stay under the cursor, but instead moves by unexpected distances.
Looks like the data zoom slider has a momentum/coordinate mapping issue. Bumping this up the priority list since multiple people are hitting it.
Here’s a demo of wip rendering engine we’re working on that boosted our previous capabilities of 10M data points to 100M data points.
https://x.com/TapeSurfApp/status/2009654004893339903?s=20
[0]: https://chartgpu.github.io/ChartGPU/examples/live-streaming/...
[1]: https://crisislab-timeline.pages.dev/examples/live-with-plug...
[1]: https://github.com/ChartGPU/ChartGPU/blob/main/.cursor/agent...
[2]: https://github.com/ChartGPU/ChartGPU/blob/main/.claude/agent...
- new account
- spamming the project to HN, reddit etc the moment the demo half works
- single contributor repo
- Huge commits minutes apart
- repo is less than a week old (sometimes literally hours)
- half the commits start with "Enhance"
- flashly demo that hides issues immediately obvious to experts in the field
- author has slop AI project(s)
OP uses more than one branch so he's more sophisticated than most.
There doesn't seem to be a communication mechanism that has minimal memcopy or no serialization/deserialization, the security boundary makes this difficult.
I have a backend array of 10M i16 points, I want to get this into the frontend (with scale & offset data provided via side channel to the compute shader).
As it stands, I currently process on the backend and send the frontend a bitmap or simplified SVG. I'm curious to know about the opposite approach.
https://plotly.com/python/performance/
Haha, Highcharts is a running joke around my office because of this. Every few years the business will bring in consultants to build some interface for us, and every time we will have to explain to them that highcharts, even with it's turbo mode enabled chokes on our data streams almost immediately.
After the initial setup and learning curve, it was actually very easy. All in all, way less complicated than all the performance hacks I had to do to get 0.01% of the data to render half as smooth using d3.
Although this looks next level. I make sure all the computation happens in a single o(n) loop but the main loop still takes place on the cpu. Very well done
To anyone on the fence, GPU charting seemed crazy to me beforehand (classic overengineering) but it ends up being much simpler (and much much much smoother) than traditional charts!
[1] https://phrasing.app
How do you think is it possible? Because on RN most of the Graph Libs on CPU or Skia (which is good but still utilise CPU for Path rendering)
One more issue is that some browser and OS combinations do not support WebGPU, so we will still have to rely on existing libraries in addition to this, but it feels promising.
The slider should now track the cursor correctly on macOS. If you tried the million-points demo earlier and the zoom felt off, give it another shot.
This is why I love launching on HN - real feedback from people actually trying the demos. Keep it coming! :)
Drawing and scrolling live data was problem for a lib (dont remember which one) because it was drawing the whole thing on every frame.
Although dragging the slider at the bottom is currently kind of broken as mentioned in another comment, seems like they are working on it though.
However, this is pretty great; there really aren't that many use cases that require more than a million points. You might finally unseat dygraphs as the gold standard in this space.
I guess the real draw here is smooth scrolling and zooming, which is hard to do with server-rendered tiles. There's also the case of fully local use, where server rendering doesn't make much sense.
The computer on my desk only costs me the electric power to run it, and there's 0 network latency between it and the monitor on which I'm viewing charts. If I am visualizing some data and I want to rapidly iterate on the visualization or interact with it, there's no more ideal place for the data to reside than right there. DDR5 and GPUs will be cheap again, some day.
I agree, unfortunately no library I've found supports this. I currently SSR plots to SVG using observable plot and JSDom [0]. This means there is no javascript bundle, but also no interactivity, and observable doesn't have a method to generate a small JS sidecar to add interactivity. I suppose you could progressive enhance, but plot is dozens of kilobytes that I'd frankly rather not send.
[0] https://github.com/boehs/site/blob/master/conf/templating/ma...
I think it's just a different mindset; GIS libs like Leaflet kind of assume they're the centerpiece of the app and can dictate a bunch of structure around how things are going to work, whereas charting libs benefit a lot more from "just add me to your webpack bundle and call one function with an array and a div ID, I promise not to cause a bunch of integration pain!"
Last time I tried to use it for dashboarding, I found Kibana did extremely aggressive down-sampling to the point that it was averaging out the actual extremes in the data that I needed to see.
It’s more low level than a full charting library, but most of it can run natively on the server with zero config.
I’ve always found performance to be kind of a drag with server side dom implementations.
Funny enough, I am doing something very similar: a C++ portable (Windows, Linux MacOS) charting library, that also compile to WASM and runs in the browser...
I am still at day 2, so see you in 3 days, I guess!
Error message: "WebGPU Error: Failed to request WebGPU adapter. No compatible adapter found. This may occur if no GPU is available or WebGPU is disabled.".
WebGPU is supported on Chrome and on the latest version of Safari. On Linux with all browsers WebGPU is only supported via an experimental flag.
Maybe I messed with the settings at some point and disabled something.
Vega/VGlite have amazing charting expressivity in their spec language, most other charting libs don't come close. It would be very cool to be able to take advantage of that.
[0] https://github.com/vega/vega-webgl-renderer
https://caniuse.com/webgpu
Please support a fallback, ideally a 2D one too. WebGPU and WebGL are a privacy nightmare and the former is also highly experimental. I don't mind sub-60 FPS rendering, but I'd hate having to enable either of them just to see charts if websites were to adopt this library.
The web is already bad requiring JavaScript to merely render text and images. Let's not make it any worse.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1870699
And there is the issue of getting the browser to use the correct GPU in the first place, but that is a different can of worms.
Biggest issue is MacOS users with newer Safari on older MacOS.
This blocks progress (and motivation) on some of my projects.
But personally, I'm not going to start turning on unsafe things in my browser so I can see the demo. I tried firefox and chromium and neither worked so pfft, whatever.
Maybe am just bad at reading specifications or finding the right web browser.
Would be great if you had a button there one can press, and it does a 10-15 second benchmark then print a min/max report, maybe could even include loading/unloading the data in there too, so we get some ranges that are easier to share, and can compare easier between machines :)
Love the benchmark button idea. A "Run Benchmark" mode that captures: - Load time - GPU time - CPU time - Min/max/avg FPS over 10-15 seconds - Hardware info
Then export a shareable summary or even a URL with encoded results. Would make for great comparison threads.
Adding this to the roadmap - would make a great v0.2 feature. Thanks for the suggestion!
I've been looking for a followup to uPlot - Lee who made uPlot is a genius and that tool is so powerful, however I need OffscreenCanvas running charts 100% in worker threads. Can ChartGPU support this?
I started Opus 4.5 rewrite of uPlot to decouple it from DOM reliance, but your project is another level of genius.
I hope there is consideration for running your library 100% in a worker thread ( the data munging pre-chart is very heavy in our case )
Again, congrats!
Worker thread support via OffscreenCanvas is a great idea and WebGPU does support it. I haven't tested ChartGPU in a worker context yet, but the architecture should be compatible - we don't rely on DOM for rendering, only for the HTML overlay elements (tooltips, axis labels, legend).
The main work would be: 1. Passing the OffscreenCanvas to the worker 2. Moving the tooltip/label rendering to message-passing or a separate DOM layer
For your use case with heavy data munging, you could also run just the data processing in a worker and pass the processed arrays to ChartGPU on the main thread - that might be a quicker win.
Would you open an issue on GitHub? I'd love to understand your specific workload better. This feels like a v0.2 feature worth prioritizing.
There is certainly something beautiful about your charging GPU code being part of a file that runs completely isolated in another thread along with our websocket Data fire hose
Architecturally that could be something interesting where you expose a typed API wrapping postmessage where consumers wanting to bind the main thread to a worker thread could provide the offscreen canvas as well as a stream of normalized, touch and pointer events, keyboard and wheel. Then in your worker listeners could handle these incoming events and treat them as if they were direct from the event listeners on the main thread; effectively, your library is thread agnostic.
I'd be happy to discuss this on GitHub. I'll try to get to that today. See you there.
On the topic of support for worker threads, in my current project I have multiple data sources, each handled by its own worker. Copying data between worker and main thread - even processed - can be an expensive operation. Avoiding it can further help with performance.
3D is coming (it's the same rendering pipeline), but I'd want to get the 2D story solid first before expanding scope.
The slice animation is doable though - we already have animation infrastructure for transitions. An "explode slice on click" effect would be a fun addition to the pie/donut charts.
What's your use case? Dashboard visuals or something else?
Even when I turn on dom.webgpu.enabled, I still get "WebGPU is disabled by blocklist" even though your domain is not in the blocklist, and even if I turn on gfx.webgpu.ignore-blocklist.
Very cool project. Thanks!!!
https://caniuse.com/webgpu
I hope you have a way to monetize/productize this, because this has three.js potential. I love this. Keep goin! And make it safe (a way to fund, don't overextend via OSS). Good luck, bud.
Also, you are a master of naming. ChartGPU is a great name, lol!
Interesting you mention three.js - there's definitely overlap in the WebGPU graphics space. My focus is specifically on 2D data visualization (time series, financial charts, dashboards), but I could see the rendering patterns being useful elsewhere.
On sustainability - still figuring that out. For now it's a passion project, but I've thought about a "pro" tier for enterprise features (real-time collaboration, premium chart types) while keeping the core MIT forever. Open to ideas if you have thoughts.
Appreciate the kind words! :)
Off the top of my head, look into Order Book Heatmaps, 3D Volatility Surfaces, Footprint Charts/Volatility deltas. Integrating drawing tools like Fibonacci Retracements, Gann Fans etc. It would make it very attractive to people willing to pay.
Which demo were you on? (million-points, live-streaming, or sampling?) I'll test on M1 today and get a fix out.
Really appreciate you taking the time to try it :)
Most high-level charting libraries already support downsampling. Rendering data that is not visible is a waste of CPU cycles anyway. This type of optimization is very common in 3D game engines.
Also, modern CPUs can handle rendering of even complex 2D graphs quite well. The insanely complex frontend stacks and libraries, a gazillion ads and trackers, etc., are a much larger overhead than rendering some interactive charts in a canvas.
I can see GPU rendering being useful for applications where real-time updates are critical, and you're showing dozens of them on screen at once, in e.g. live trading. But then again, such applications won't rely on browsers and web tech anyway.
The fact that the code was generated by a human or a machine is less and less important.
Sorry if this is weird, it's just I've never personally experienced a comment with anything more than 100 - 200 points. And that was RARE. I totally get if you don't want to...but like, what were your "kilopoint" comments, or thereabouts? </offtopic>
So, apparently these are my five most upvoted comments (based on going through the first 100 pages of my own comments...):
- 238 - https://news.ycombinator.com/item?id=46574664 - Story: Don't fall into the anti-AI hype
- 127 - https://news.ycombinator.com/item?id=46114263 - Story: Mozilla's latest quagmire
- 92 - https://news.ycombinator.com/item?id=45900337 - Story: Yt-dlp: External JavaScript runtime now required f...
- 78 - https://news.ycombinator.com/item?id=46056395 - Story: I don't care how well your "AI" works
- 73 - https://news.ycombinator.com/item?id=46635212 - Story: The Palantir app helping ICE raids in Minneapolis
I think if you too retire, have nothing specific to do for some months, and have too much free time to discuss with strangers on the internet about a wide range of topics you ideally have strong opinions about, you too can get more HN karma than you know what to do with :)
The idea that GPU vendors are going to care about memory access violations over raw performance is absurd.
What is wrong with you JavaScript bros.