How To Craft A Blazingly Fast Graphics Device

2024/12/07 Japan.R 2024

Hiroaki Yutani

“R is slow”

I usually don’t say these words.
I’m fine if it’s decently fast. But…

Rendering a single plot is fine

Making animation is too slow!

If one plot takes 0.1s, a 10s of animation at 60fps will take…

\[ 0.1 \times 10 \times 60 = 1 [\texttt{minutes}] \]

One minute for every single tweak on one of the many parameters!??

Short summary

I need a blazingly fast graphics device!

What is
“Graphics Device”?

What is “Graphics Device”?

What actually renders your plot
What is opened by such functions:
- png()
- pdf()
- ragg::agg_png()
- svglite::svglite()
- …

What is “Graphics Device”?

Even if you don’t know the word, most of you probably know how to use it.

# open a graphics device
ragg::agg_png()

# plot something
ggplot(mpg, aes(displ, hwy, colour = class)) + 
  geom_point()

# a cryptic spell that gives you a PNG!
dev.off()

Graphics Device API

C API
Primitive instruction like “draw a line”
A user doesn’t call this directly; instead,
- graphics package
- grid package
- some package on top of either one (e.g. ggplot2)

Examples of the API

Draw a line:

void (*line)(double x1, double y1,
  double x2, double y2,
  const pGEcontext gc, pDevDesc dd);

Draw a text:

void (*textUTF8)(double x, double y,
  const char *str, double rot,
  double hadj, const pGEcontext gc, pDevDesc dd);

Calling `rect()` API

Calling `circle()` API

Calling `polyline()` API

Calling `textUTF8()` API

Done!

Graphics Device API

A complex plot is a composite of many simple graphical instructions.
Graphics device API is such low-level graphical instructions.
A graphics device is what implements the graphics device API.

Vellogd

GPU-powered graphics device
Heavily relies on the vello Rust crate

https://github.com/yutannihilation/vellogd-r

Vello

2D graphics rendering engine
GPU-accelerated via WebGPU
Cross-platform (macOS, Windows, Linux, iOS, etc)

https://github.com/linebender/vello

WebGPU

API for performing operations on GPU
Portable and safe
Unlike the name indicates, WebGPU is not only for web!!

Disclaimer

I’m not sure if using GPU is faster in the case of rendering a single plot.
- Passing data to GPU can be costly
- Graphics Device API is not designed as GPU-friendly (discuss later)
But, it is worth a try at least!

It works! (to some extent…)

So, is this fast?

Let’s measure!

A simple example code taken from ggplot2’s README:

ggplot(mpg, aes(displ, hwy, colour = class)) +
  geom_point()

Result

The elapsed times of API calls:

...::size: 302 times (0.471 ms)
...::circle: 241 times (0.100 ms)
...::text_width: 48 times (0.084 ms)
...::char_metric: 31 times (0.081 ms)
...::activate: 1 times (0.035 ms)
...::text: 19 times (0.034 ms)
...::polyline: 28 times (0.021 ms)
...::new_page: 1 times (0.007 ms)
...::rect: 10 times (0.004 ms)

Result

The elapsed times of API calls:

...::size: 302 times (0.471 ms)
...::circle: 241 times (0.100 ms)
...::text_width: 48 times (0.084 ms)
...::char_metric: 31 times (0.081 ms)
...::activate: 1 times (0.035 ms)
...::text: 19 times (0.034 ms)
...::polyline: 28 times (0.021 ms)
...::new_page: 1 times (0.007 ms)
...::rect: 10 times (0.004 ms)

Total time:

724.536 ms

Result

The device is fast
But, it matters verrry little…

Why this slow?

Probably “724 ms” is very inaccurate; my measurement was too lazy.
Still, it’s true that it takes much more time to translate into the graphics device APIs than to call the APIs!

grid

A part of base R
An alternative to R’s standard graphics system
Provides the functionality of the flexible layout of graphics elements

gtable

A layout engine
Works on top of {grid}

Layout is hard

A position cannot be determined without the relationship between other elements
A layout engine doesn’t know
- window size
- font-related information
Layouts can be nested deeply

To determine the position of this,

the window size matters

the sizes of the neighbors matter

But, why does this need to be re-calculated on every plot??

Can’t we assume the window size is fixed?
Can’t we cache the title, legend, and axis?

Typically, most of the changes happen in this area

Limitations

There’s no such API!
As there are various graphics devices, the API needs to be conservative

But, what if…

we bypass the API??

e.g. Animation API

Lottie

An open-source JSON format for vector graphics animation
Exported from animation softwares like Adobe After Effects
Hosted with the Linux Foundation

https://lottie.github.io/

Lottie

Lottie is about portability

Is it portable to R??

Yes!

vellogd()

ggplot(mpg, aes(displ, hwy, colour = class)) + 
  geom_point()

# add 🐯
add_lottie_animation("tiger.json")

Yes!

Vello rocks

But, how is this useful?

R can generate Lottie (in theory)

The spec is openly available
Text format (JSON)

R can generate Lottie (in theory)

“in theory”

= I don’t implement it yet :P

Instead of rendering one by one

serialize the animation

+ non animated elements

Doesn’t this look simpler?

Because this does heavy lifting!

Let
the graphics device do heavy lifting

Examples of what vello can offer but there’s no corresponding API

Animation
Bezier curve
Rich text formatting like text-wrapping

Bezier curve

No graphics device API
grid::grid.bezier() can draw a bezier curve by flattening the curve to small lines
This works, but is inefficient for GPU

Rich text formatting

{marquee} should solve most of the problems
Still, I’m wondering if this is a graphics device’s job
- Modern 2D rendering engine is so powerful; almost a web browser!

So what?

By taking care of common tasks, the graphics device can be language-agnostic

A language-agnostic device?

Is a language-agnostic graphics device useful?

Honestly, I’m not sure!
A device can’t replace higher interfaces; we still can’t live without such libraries as ggplot2 and matplotlib.
But, it can be a foundation for language-agnostic data visualization tool.

Summary