White labelling interactives

One ambition we’ve had in the team is to allow white labelling of our interactives so external organisations can style it how they like. From conversations with news organisations, this is something they would like and would help with them embedding our tools.

We were inspired when Jon McClure came to speak to us about the graphics team at Reuters. Their business model also relied on getting other organisation to restyle their interactives as a way of selling their embeds.

We found the 2022 US mid term election results page on the BBC as an example.

The map and table are from Reuters but BBC have styled them with their own colours and fonts.

Inspecting the element you can see that the iframe has some parameters including one where a URL is given for an externally hosted CSS.

customStylesheet=https://static.files.bbci.co.uk/elections/styles/BBC_Override.css

I had a chat with Ahmad, the brains behind ONS’ svelte components and he came up with a way to integrate it with our components.

Again following the Reuter’s way of doing things we created a prop for our Theme component, a allowClientOverrides boolean which can then be targetted with CSS. By using CSS variables, these overrides are carried through to child components.

How it works

When using the <Theme> component, specify a theme and set the prop allowClientOverrides to true. This then sets a div with the class client-css-override inside the theme which can then be targetting with CSS, for example

div.client-css-override{
    --text: #f9ea9a;
}

The external stylesheet is loaded if customStylesheet is passed as a parameter in the url. SvelteKit uses the $page store to get the URL and then gets the key customStylesheet.

const customStylesheet = $page.url.searchParams.get("customStylesheet");

If there is a customStylesheet it gets loaded via a <link> element.

{#if customStylesheet}
       <link rel="stylesheet" href={customStylesheet} />
{/if}

Here’s a little demo.

Scrollystarter with .docx and ArchieML parser

One problem we’ve encounted while producing an article that uses a scrollytelling format, for example Understanding unemployment: what role does ethnicity and disability play?, is editing the text.

The problem: word and html

The copy for the article is normally drafted in a word doc. The text is then copied across from the word doc into HTML and Svelte elements. This involved a lot of hand-cranking, for example open and closing <p> tags or <Em> component. Hyperlinks also need the href not just the text to display.

The word doc for the article is sent round for review and people add comments, track changes, or don’t track changes and just edit it.

When you are trying to keep the text in the scrolly up to date with the changes, it becomes a lot of work to track which changes you need to implement, or spending lots of time putting all the text in fresh again.

The solution: ArchieML

The New York Times created a markup language called ArchieML which was built for the newsroom. It relies on everything that needs to put into a structure having a key:value pair. And anything else is just a comment.

Reuters and others have a workflow where they draft in Google docs, writing their content in ArchieML. This then gets pulled into a SvelteKit alongside their graphics components so they can quickly spin up a scrollytelling article.

Making it work with .docx

Unfortunately at ONS, we can’t use Google docs so I thought can we do it with word’s .docx files.

A quick search showed Mammoth as a good node library to convert .docx to HTML, passed through a HTML parser and then finally parsed as ArchieML, which is the approach suggested by ArchieML in their example script when working with Google docs. Converting to HTML preserves hyperlinks but you don’t really need anything else.

Using AI

I could get Mammoth to read the file, but when I passed that to a HTML parser, it was harder to figure out what was going on. I could tell it was nesting stuff, but not much else. It looked something like this.

<ref *1> [
  <ref *2> Element {
    parent: Document {
      parent: null,
      prev: null,
      next: null,
      startIndex: null,
      endIndex: null,
      children: [Circular *1],
      type: 'root'
    },
    prev: null,
    next: Element {
      parent: [Document],
      prev: [Circular *2],
      next: [Element],
      startIndex: null,
      endIndex: null,
      children: [Array],
      name: 'p',
      attribs: {},
      type: 'tag'
    },
    startIndex: null,
    endIndex: null,
    children: [ [Text] ],
    name: 'p',
    attribs: {},
    type: 'tag'
  },
  <ref *3> Element {
    parent: Document {
      parent: null,
      prev: null,
      next: null,
      startIndex: null,
      endIndex: null,
      children: [Circular *1],
      type: 'root'
    },
    prev: <ref *2> Element {
      parent: [Document],
      prev: null,
      next: [Circular *3],
      startIndex: null,
      endIndex: null,
      children: [Array],
      name: 'p',
      attribs: {},
      type: 'tag'
    },
    next: Element {
      parent: [Document],
      prev: [Circular *3],
      next: [Element],
      startIndex: null,
      endIndex: null,
      children: [Array],
      name: 'p',
      attribs: {},
      type: 'tag'
    },
    startIndex: null,
    endIndex: null,
    children: [ [Text] ],
    name: 'p',
    attribs: {},
    type: 'tag'
  },

  
...

  <ref *17> Element {
    parent: Document {
      parent: null,
      prev: null,
      next: null,
      startIndex: null,
      endIndex: null,
      children: [Circular *1],
      type: 'root'
    },
    prev: <ref *16> Element {
      parent: [Document],
      prev: [Element],
      next: [Circular *17],
      startIndex: null,
      endIndex: null,
      children: [Array],
      name: 'p',
      attribs: {},
      type: 'tag'
    },
    next: null,
    startIndex: null,
    endIndex: null,
    children: [ [Text] ],
    name: 'p',
    attribs: {},
    type: 'tag'
  }
]

In the example script from ArchieML, there’s a bit to specify how to handle the parsing, so I knew I had to adapt it. In the example script, it basically makes most things text apart from <a> tags.

I used the Google AI Bard to help me write a function to go through the parsed output and extract the text from the word docx.

This could then be passed again to ArchieML to be parsed into JSON. Finally I had a workflow that worked.

Making a scrolly starter

Modelled on the robo-embed template, I wrapped everything up into a scrolly starter template that uses ONS Svelte Component’s feature article as an example for the content. It takes the content from a demo .docx written ArchieML. More instruction of how to use it are in the repo.

Future improvements

At the moment, it uses svelte’s @html to render the content as html so that links work. However, custom components like <Em> don’t work so I can’t make it exactly like the feature article.

Making web components in svelte 4

I came across this BBC article where they have a scrolly inside a webpage and looking through the DOM, they are using a <template> and using the template as a web-component which then inserts stuff onto a shadow-root. Here’s a good MDN article about templates and slots. I first heard about web-components when datawrapper said they had a new way of embedding charts using web-components.

When we tried to do a scrolly embedded in an article using an iframe, the page and iframe scrolly separately so was a bit janky and quite an unsatisfying experience. Since we use Svelte as our main framework, I did some exploring and found Svelte can compile components to web components.

There is a way of using the old svelte template and rollup to generate web-components that can be imported as standalone .js files (See phptuts or logrocket for a walkthrough) but this version uses vite and svelte 4.

The how

Svelte can set up projects to be like a component library using npm create svelte@latest and this is the recommended way from comments.

For library projects, your interesting stuff is in lib and then how it works is in routes. I’ve created a svelte component inside lib and added this line to the to the .svelte file.

<svelte:options customElement="count-er> 

Also added compilerOptions to svelte.config.js

compilerOptions:{customElement:true}

When you run npm run package, this creates files in a dist folder.

Following instructions from this recipe you can run vite again on the dist/index.js to create the standalone iife .js files. See the vite.webcomponents.config.js and the modified build command in the package.json file. These standalone files can be then loaded from a webpage and then the web-component can be used. See the test-page folder for an example.

The finish result

Here’s my github repo for svelte 4 web components. I used this template as a good starting point and followed this recipe which I found from a stackoverflow answer.

Modifying svelte charts

Really quick post about modifying modules from an NPM import. I wanted to use the svelte charts library but adapt it. I tried editing the files in the node_modules folders but looks like they aren’t watched. Followed a few stackoverflow trails (this comment and this answer).

What you have to do is download the svelte chart repo locally and then from the project you’re working in tell npm to install it from a local dependency.

npm install --save /path/to/localversion

You’ll have to run npm install from the local folder (svelte-charts) to get all the stuff it relies on.

In the package.json it now lists the path of svelte-charts as a local one

"devDependencies": { "@onsvisual/svelte-charts": "file:../modified-svelte-charts",

Household composition by single year of age

A little while ago Patrick Scott spotted this interesting graphic about the UK population by household type thinking it would be great over time.

He also thought it would fit well into a scrolly format, similar to this article looking at Donald Trump’s finances from the New York Times.

Ahmad’s been working on svelte-components and I thought it would be a good time to try it out. Ahmad and Andrew have been working on using storybook.js to make it more usable with documentation of how to use the different component. There’s a scrolly in the template so it was just a case of adapting that.

However I did come across an issue when I was trying to implement the area chart. The way svelte-charts handles transitions is that it expects all the data to be present and it can’t handle enter-exit.

After a conversation with Ahmad about how best to handle this, we agreed to just set all the series to be zero except the first area and then transition to numbers as you scrolled so the lines are all there to start with.

And here’s the scrolly about household type in action and source code.

I got the data from the Create a custom dataset service where you can select different geographies and combinations of different variables at different levels of details. For this I chose England and Wales, all 101 ages and all 12 household types link to dataset.

I had to run some R code to condense the 12 categories down to something similar to the original image.

df %>%
mutate(hhtype=case_when(
Household.type..12.categories..Code == 2 | Household.type..12.categories..Code==4 | Household.type..12.categories..Code == 6~ "Couple: Dependent children",
Household.type..12.categories..Code == 3 | Household.type..12.categories..Code == 5 | Household.type..12.categories..Code ==7 ~ "Couple: No dependent children",
Household.type..12.categories..Code == 1 ~ "One-person household",
Household.type..12.categories..Code == 8 ~ "Lone parent: Dependent children",
Household.type..12.categories..Code == 9 ~ "Lone parent household: No dependent children",
Household.type..12.categories..Code == 10 ~ "Multi-person household: All students",
Household.type..12.categories..Code == 11 ~ "Multi-person household: Other",
.default = "Does not apply"
))


df2%>%group_by(Age..101.categories..Code,hhtype) %>% summarise(obs=sum(Observation))

In the svelte code there are a couple of interesting bits to pull out. One is how it handles what data to present. Data for svelte chart is in a tidy data format and we want to filter multiple categories. I found an easy way of filtering by including the series to filter in an array and using this method as set out in this stackoverflow answer.

The generic format is data.filter(d=>[array of series to filter by].includes(d.series)). Previous I had said that we had to set the values to 0 if we don’t want the line to be seen as it needs to be there so it doesn’t enter in but not seen. We do this with a .map and if the series is in the ones to be included we return all the data, otherwise return 0. This is how it’s handled in the demo.

linechartdata = data.places.map(d=>{if(filterfuncs[e.detail.index].includes(d.hhtype)){return d}else{return {age:d.age,obs:0,hhtype:d.hhtype}}})

It’s a bit hard coded to set it to zero and is a funny way round svelte charts but it works.