A/B Split Testing Using Vue 3 and Plausible Analytics

Split Testing (aka A/B Testing) is an important technique for validating hypothesis and maximizing conversions. Read how to easily use and track split tests in your Vue app/website.

Users will randomly see option A or B and you can benchmark the conversions for each. When you have to decide between two possible options, or want to verify that an update to your website or app is an improvement after all, you should always perform a split test.

Requirements

In this short guide we’re using Vue3 with Pinia and TypeScript for cross-component state management and Plausible Analytics, an open-source and privacy-friendly analytics service, that even works without cookie or DSGVO consents. You can either self-host Plausible or use their paid hosting service. Of course, you can also use different services (for B2B solutions I can highly recommend june.so) and replace the corresponding tracking code.

Example Scenario

In the following code we’re assuming that we have a product marketing page and want to benchmark different wordings (or layouts/images). Our conversion we use for benchmarking both variants is the click on a „Sign up“ button.

Plausible Event Tracking Setup

1. In Plausible, click on your domain name and select „Site settings“
2. Switch to the „Goals“ tab and click the „Add goal“ button
3. Select „Custom event“ as the goal trigger and enter an event/conversion name, that you want to use for benchmarking your split test. Given our example scenario, we’ll just use „Signup“ here.
4. That’s it! Plausible is now configured to track and display the event we created.

Pinia Store Setup for our Split Tests

We want the selected split test variants to be persistent across our whole website/app, so we’ll use Pinia for shared state management. For that we’ll create a new file splitTests.ts in src/stores. We also want our store to support multiple concurrent tests, in case we use multiple variants across our app later.

With the following code, we create a new Pinia store that contains an object that is going to contain the selected variants. Moreover, we define a method split that is used for (once) randomly selecting and returning a variant.

import { ref } from 'vue'
import { defineStore } from 'pinia'

export const useSplitTestsStore = defineStore('splitTests', () => {
    const splits = ref<{ [key: string]: string }>({})

    const split = (key: string): string => {
        if (!splits.value[key]) {
            splits.value[key] = Math.random() < 0.5 ? 'A' : 'B'
        }

        console.info('split test', key, splits.value[key])
        return splits.value[key]
    }

    return { split }
})

We can now integrate our Pinia store with the following code in our landing page’s view/component:

<template>
    <div v-if="landingPageVariant === 'A'">
        <h1>Variant A</h1>
    </div>
    <div v-else>
        <h1>Variant B</h1>
    </v-else>
</template>

<script lang="ts" setup>
import { useSplitTestsStore } from '@/stores/splitTests'

// initialize our Pinia store
const splitTests = useSplitTestsStore()

// create a split test named "landingPage"
const landingPageVariant = splitTests.split('landingPage')
</script>

Although our store’s state is now persistent while navigating through our app and across components, the active variants might change when reloading our website. As we don’t want a single user to see different variants, we need to persist the selection using the local storage.

In the setup method of our Pinia store (after the definition of our splits object) we’ll restore the store’s state (if set in local storage) and add another method for persisting our state:

...
const splits = ref<{ [key: string]: string }>({})
const restored = window.localStorage.getItem('splitTests')

if (restored) {
    try {
        splits.value = JSON.parse(restored) || {}
    } catch (e) {
        console.warn('error restoring split testing data', e)
    }
}

const persist = () => {
    window.localStorage.setItem('splitTests', JSON.stringify(splits.value))
}

In our split(key: string) method we only need to call the persist method after a new variant is picked:

splits.value[key] = Math.random() < 0.5 ? 'A' : 'B'
persist() // <--

Add Plausible Tracking to our Store

After adding the tracking snippet (that plausible shows you after creating a website) in the index.html you'll need to add another snippet that registers the custom event tracking handler. At the bottom of the <head> block, add the following line:

<script>window.plausible = window.plausible || function () { (window.plausible.q = window.plausible.q || []).push(arguments) }</script>

Now we can call window.plausible('eventName') anywhere in our application to trigger a custom event. Moreover, the method accepts a second parameter that we can use to pass additional properties along our event. We're going to use these props to add our split test variant.

In the landing page’s component (or wherever the Signup button is located), create an event listener with the following code:

// @ts-ignore
window.plausible(
    'Signup',
    {
        props: {
            landingPageVariant: useSplitTestsStore().split('landingPage'),
        },
    },
)

As soon as events were captured, you will see the „Signup“ goal in Plausible. Click on the event to see the parameters breakdown and compare your variants with each other. This will look as follows:

However, as our Math.random call that determines the selected variant doesn’t guarantee an exact 50:50 distribution, we should also track the selected variants. In our Pinia store’s split() method add the following tracking code after our line with Math.random:

// @ts-ignore
window.plausible(`${key} selection`, { props: { variant: splits.value[key] } })

In our example this would trigger the goal „landingPageVariant selection“ with the variant prop A or B. Add this goal name in the plausible settings, and you can see how often which variant was selected and consider this when benchmarking the conversion numbers.

Final Code

After completing the steps above, your code should look similar to this:

src/stores/splitTests.ts:

import { ref } from 'vue'
import { defineStore } from 'pinia'

export const useSplitTestsStore = defineStore('splitTests', () => {
    const splits = ref<{ [key: string]: string }>({})
    const restored = window.localStorage.getItem('splitTests')

    if (restored) {
        try {
            splits.value = JSON.parse(restored) || {}
        } catch (e) {
            console.warn('error restoring split testing data', e)
        }
    }

    const persist = () => {
        window.localStorage.setItem('splitTests', JSON.stringify(splits.value))
    }

    const split = (key: string): string => {
        if (!splits.value[key]) {
            splits.value[key] = Math.random() < 0.5 ? 'A' : 'B'
            persist()

            // @ts-ignore
            window.plausible(`${key} selection`, { props: { variant: splits.value[key] } })
        }

        console.info('split test selected', key, splits.value[key])
        return splits.value[key]
    }

    return { split }
})

Our landing page component

<template>
    <div v-if="landingPageVariant === 'A'">
        Variant A
    </div>
    <div v-else>
        Variant B
    </v-else>
    <button @click="signUpClick()">Sign Up</button>
</template>

<script lang="ts" setup>
import { useSplitTestsStore } from '@/stores/splitTests'

// initialize our Pinia store
const splitTests = useSplitTestsStore()

// create a split test named "landingPage"
const landingPageVariant = splitTests.split('landingPage')

const signUpClick = () => {
    // @ts-ignore
    window.plausible(
        'Signup',
        {
            props: {
                landingPageVariant,
            },
        },
    )
}
</script>