All pages Environment API reference

Documentation Build a light show

Environment API reference

Everything an environment package can declare, receive and import. This is version 1 of the environment API.

This is the reference for people writing a light show environment. Create an environment is the walkthrough, and How the light show works explains where the data comes from.

Within one API version, things are only ever added. A package written for version 1 keeps working on every Rainy that speaks version 1.

The package

A package is a .zip file:

my-show.zip
  rainy-environment.json    the manifest
  index.js                  the entry module
  style.css                 optional
  preview.jpg               the picture people choose it by
  …                         your own modules, libraries, shaders, images, fonts

The manifest sits at the root of the zip, or inside a single top-level folder, which is what a repository download gives you.

Limit
Size 20 MB zipped, 60 MB unpacked, 500 files.
File types js mjs json css wasm; images png jpg jpeg webp avif gif svg; fonts woff2 woff ttf otf; shaders and text glsl vert frag wgsl txt md; data bin ktx2 hdr; media mp3 ogg mp4 webm.
Skipped Hidden files, node_modules, and what archivers add (__MACOSX, .DS_Store).
Refused Absolute paths, .., links, other file types, and anything outside the limits.

The manifest

rainy-environment.json:

Field
manifest required Version of this file's format: 1.
api required Version of the environment API the code was written for: 1. Rainy refuses a package written for a version it does not speak.
id required 2 to 32 characters: lowercase letters, digits and hyphens, starting with a letter. Installing a package with the id of an installed one replaces it. classic is taken.
name required Shown in Settings and in the show. Up to 60 characters.
version required Looks like 1.2.3.
entry required The module to load: a .js or .mjs file of the package.
description One line for the picker. Up to 300 characters.
author, license Shown in Settings.
homepage An http or https address.
styles Up to 8 stylesheets, loaded while the environment is on stage and removed when it leaves. See Styling the player.
preview recommended The picture in Settings: a 16:9 image, 960 × 540 is plenty. Without one the environment gets a blank plate.
layout stage (default) or immersive. See Layouts.
requires Any of webgl, webgl2, webgpu. On a device without one of them, the environment cannot be picked.
i18n name and description per language code: { "es": { "description": "…" } }.

The environment class

The entry module's default export is a class. Rainy creates one instance when the environment goes on stage. Only the first four methods are required.

Method When
mount(ctx) Once, to build everything. May return a promise.
update(S) Every frame, first: read the show, fire cues, move the lyrics.
render(S) Every frame, second: draw.
unmount() Once, to release what the page cannot release for you: WebGL contexts, workers, timers.
resize(cssW, cssH, { tier, lowPower, scale }) Every frame, before update. Return early when nothing changed. Render at cssW × scale by cssH × scale. tier is 0 (full), 1 or 2 (lightest).
setScore(score) The song's score arrived, or went away (null).
setTrack(track) { id, title, artist, cover } of the song on stage, or null. cover is an image address or null.
onTrackChange() Another song started.
leave() Another environment is taking over. Your layer is already fading out.
dropLyrics() Take the lyrics off stage now.

When an environment fails

Rainy takes the environment off stage, removes everything it added and continues in Classic when:

  • update or render throws three frames in a row,
  • any other method throws,
  • loading and mounting take more than ten seconds, or
  • the environment calls ctx.fail(error).

It stays off on that device until the page is reloaded or the package is updated. The error is logged to the browser's console.

ctx

What mount receives.

ctx.root Your own layer over the stage: an empty element that fills it. Canvases and DOM go in here. Rainy fades it in after mount, fades it out when you leave, and removes it. It ignores the pointer unless you set pointer-events on something inside it.
ctx.lyrics The lyrics timeline, also passed every frame as S.lyrics.
ctx.asset(path) The address of a file of your package. Relative imports and url() in your stylesheets also work as usual.
ctx.ui.safeArea() { top, right, bottom, left }, each 0 to 1: how much of that side of the stage the player covers right now.
ctx.ui.addButton({ icon, title, onClick, pressed }) A button of your own in the show's bar. icon is SVG markup, 14 pixels. Returns { el, setPressed(on), remove() }.
ctx.ui.toast(message) A short notice.
ctx.ui.fontFamily The interface's font, for text you draw yourself.
ctx.locale Language of the interface: en, ca, es or pl.
ctx.id, ctx.manifest Your id and your manifest.
ctx.api The API version of this Rainy.
ctx.fail(error) You cannot go on. Rainy falls back to Classic.

@rainy/lightshow

The one module of Rainy's that a package imports.

Export
API_VERSION 1.
TAU, clamp(v, a, b), lerp(a, b, t), smooth(t) Maths. smooth is a smoothstep from 0 to 1.
hsl(h, s, l), mixHue(a, b, t), mixRgb(a, b, t), toWhite(c, t) Colour. A colour is [r, g, b], each 0 to 255.
rgba(c, alpha) A colour as a CSS string.
paletteFor({ accentHue, key, mode, label, genre, swap }) The palette of a section, as the conductor computes it. For when you need a section's colours before it plays.
stepEase(beat), altBeat(beat) Motion locked to the beat: a value that steps on each beat with a short glide, and one that swings from side to side on alternate beats.
hash(n) A repeatable pseudo-random number from 0 to 1.
floorIndex(sortedArray, t) Index of the last element that is at most t, or -1. For the score's time arrays.
LEAD, HOLD_MAX, REST_AFTER, HELD The lyric timings every environment shares, in seconds.
pickTransition(F, lineIndex, last) Which of six hand-overs a lyric line gets: slam, fall, flicker, sweep, collapse or lift. It follows the music and never repeats twice in a row.
MIN_FLASH_GAP, PHRASE_BARS The conductor's limits: seconds between full flashes, and bars in a phrase.

S: the show

Passed to update and render every frame.

S.F The musical frame. See below.
S.t The song's time in seconds, smoothed and moved by the user's sync offset.
S.now, S.dt Wall clock and frame time, in seconds. For things that are not music.
S.running The song is playing. Fire one-shot cues only when it is.
S.I Intensity, 0.3 to 1.
S.pulse { k, s, h }: kick, snare and hi-hat envelopes, 0 to 1.
S.phrase Which eight-bar phrase of the section this is, from 0.
S.palette { a, b, c, w, deep }: primary, secondary, tertiary, tinted white and ambience.
S.accentHue Hue of the user's accent colour, 0 to 360.
S.dim Master dimmer, 0 to 1. Multiply your light by it.
S.flash(amount, color) Ask for a flash, 0 to 1. Limited to three full flashes a second, and softened under Reduce Flashing.
S.blind(amount) Ask for a blinder: a slower wash of white.
S.burst(amount) Ask for a burst: a boost that rings out over about half a second.
S.flashAmt, S.flashColor What to draw of the flashes this frame.
S.blinderAmt, S.burstAmt What to draw of the blinder and the burst.
S.prefs The user's settings: reduceFlashing, disableLasers, showBgBlur, lowPower (booleans) and intensity (auto, chill, balanced or hype).
S.lyrics, S.lyricsOn The lyrics timeline, and whether lyrics are on stage right now.
S.songId Id of the song on stage.

F: the music

S.F, the musical frame.

With a score, F.scripted is true and everything is exact. While a song is still being analysed, Rainy estimates the same fields from the audio: there is no look-ahead (nextIn and dropIn are Infinity), and section is a guess.

F.t The song's time in seconds.
F.bands { sub, bass, mid, high, air, rms, harm }, each 0 to 1: energy per band, loudness, and how tonal the sound is.
F.hits { k, s, h }: strength of a kick, snare or hi-hat that landed in this frame, otherwise 0.
F.beat, F.bar Beat and bar numbers, with fractions.
F.beatPhase, F.barPhase Position within the beat and the bar, 0 to 1.
F.beatInBar 0 to 3.
F.period Seconds per beat.
F.newBeat, F.newBar A beat or a bar started in this frame.
F.section { idx, label, t0, t1, key, mode, energy, density }. label is intro, verse, build, chorus, drop, breakdown or outro.
F.sectionProgress, F.sectionBar 0 to 1 through the section, and bars since it started.
F.nextSection, F.nextIn The section that follows, and seconds until it.
F.dropIn Seconds until the next drop.
F.buildProgress 0 to 1 through a build.
F.stopDepth, F.stopIn How deep into a silence the song is, 0 to 1, and seconds until the next one.
F.events What happened in this frame: { type, t }, with type one of drop, impact, stop or riser.
F.profile { genre, intensity }. genre is edm, pop, rock, hiphop, latin, chill or orchestral.
F.tempoConf How sure the analyser is of the tempo, 0 to 1.
F.jumped The user seeked, or the track changed. Snap, do not glide.
F.running The song is playing.

The score

What setScore receives: the whole song at once.

score.duration Seconds.
score.tempo, score.period Beats per minute, and seconds per beat.
score.beats, score.downbeats Times in seconds, sorted.
score.onsets.k, .s, .h Kicks, snares and hi-hats: { t, s }, two arrays of the same length holding time and strength.
score.env, score.fps score.env[band] holds one byte per frame, 0 to 255, for each band of F.bands, at score.fps frames a second.
score.envAt(band, t) The value of a band at a time, 0 to 1.
score.sections, score.sectionAt(t) The sections, as F.section.
score.events, score.nextEvent(type, t, horizon) The events, as F.events, and the next one of a type within horizon seconds.
score.beatAt(t), score.barAt(t) Beat and bar numbers at a time.
score.stopDepth(t) How deep into a silence the song is at a time.
score.profile As F.profile.

The lyrics

S.lyrics, the lyrics timeline. Worth reading when S.lyricsOn is true.

lyrics.ready The song has synced lyrics.
lyrics.lines { t0, t1, on, end, blank, words }. The line goes on stage at on, and its last word ends at end. blank marks an instrumental line.
words { text, s, e }: a word, with when it starts and ends.
lyrics.adlibs Backing vocals, kept apart from the lines: { s, e, k, words }. k is a running number, useful for alternating sides.
lyrics.aligned The word times are exact. Otherwise they are estimated from the length of the line.
lyrics.lineIndex(t) The line on stage at a time, or -1.
lyrics.resting(t, index) Nobody is singing that line right now.
lyrics.breakAt(t, index) { a, b } when an instrumental break is running, otherwise null.
lyrics.version Changes whenever the lines are replaced: a new song, or exact timings arriving.

Layouts

layout in the manifest says how the player is arranged around your layer.

stage The player as it is, with its artwork, controls and queue. The show runs behind them.
immersive The show is the whole picture. The queue folds away and a button in the bar brings it back over the show, the song and its controls become one small card at the bottom, and all of it dims while the pointer rests.

Styling the player

An environment restyles the fullscreen player with the stylesheets it lists under styles. Rainy loads them when the environment goes on stage and removes them when it leaves.

While your environment is on stage, the player has the class ls-env-<id>. Start every rule with it:

#fullscreen-player.ls-env-my-show .fs-progress-fill {
    background-color: rgb(var(--ls-a));
}

State classes

On #fullscreen-player:

Class
lightshow-on The show is running.
ls-env-<id> That environment is on stage.
ls-immersive The layout is immersive.
ls-idle Immersive only: the pointer is resting and the player is dimmed.
ls-queue-peek Immersive only: the user opened the queue over the show.
ls-lowpower Low Performance Mode is on. Drop your filters and shadows.
mode-modern, mode-standard The fullscreen mode the user chose. Modern has its own controls. Standard keeps the app's player bar at the bottom, so .fs-player-controls and the volume button are hidden.
layout-swapped The artwork is on the left and the queue on the right, which is the default. Without it they are the other way round.

Parts

Selector
.fs-backdrop The blurred album art behind everything.
.ls-env-root Your layer.
.fs-brand The Rainy mark, top left.
#fs-volume-btn, #fs-lightshow-btn, #fs-close-btn The buttons at the top.
.ls-hud, .ls-hud-btn The show's bar, with the tempo and genre, and its buttons, yours included.
.fs-content The box that holds the song and the queue.
.fs-current-info The song: artwork, title, artist and controls. In an immersive layout this is the card.
.fs-artwork-container The album art.
.fs-title, .fs-artist The song's title and artist.
.fs-player-controls The progress bar and the buttons.
.fs-progress-container, .fs-progress-bar, .fs-progress-fill, .fs-time The progress bar, its filled part, and the two times.
.fs-buttons, .fs-control-btn, .fs-control-btn.play-pause The playback buttons.
.fs-queue-container The panel with the queue and the lyrics.

Anything not in this table is internal and may change without notice.

Variables

Variable
--ls-a, --ls-b, --ls-c The primary, secondary and tertiary colour of the section that is playing, each as r g b. Same colours as S.palette.
--ls-w The tinted white, as r g b.
--ls-deep The dark ambience colour, as r g b.
--accent-primary The user's accent colour.
--text-primary, --text-secondary, --text-tertiary Rainy's text colours.
--bg-primary, --bg-secondary, --bg-tertiary, --bg-elevated Rainy's surface colours.
--border-subtle Rainy's border colour.
--radius-sm, --radius-md, --radius-lg, --radius-full Corner radii.
--spacing-xs, --spacing-sm, --spacing-md, --spacing-lg, --spacing-xl Spacing steps.
--font-family The interface's font.

Use a palette variable as rgb(var(--ls-a)) or, with transparency, rgb(var(--ls-a) / 0.5). The palette variables are set while the show runs, and glide from one section's colours to the next.

Limits

  • Keep to the fullscreen player. Your rules should not match anything outside #fullscreen-player.ls-env-<id>.
  • Do not hide or cover #fs-close-btn or #fs-lightshow-btn, and keep the playback controls usable.
  • CSS gets the palette, not the beat. Draw anything that moves with the music in your layer.

Tools

RAINY_DEV=1 RAINY_ENVIRONMENTS_DEV=<folder> Serve unpacked packages from a folder, uncached. Each is listed as an environment tagged Development, and stands in for the installed package with the same id.
python scripts/pack_environment.py <folder> [-o <out>] Check a folder as a server checks an upload and write <id>-<version>.zip, in dist by default. The same files always give the same zip.
node --import ./tests/js/sdk-hook.mjs --test <files> Run a package's own tests in Node, with @rainy/lightshow resolved as the browser resolves it.