Spotify’s got a huge catalog of music and 10 million subscribers. But what happens when the elite critics it’s co-opting for its newest feature recommend Led Zeppelin, Adele, Arcade Fire. For those allergic to apps, Spotify has long offered a web player that, while not quite as intuitive as the native desktop app, offers quick, lightweight access. Apple has followed suit, with a. Spotify on iOS Without Jailbreak and With AppValley Application. Latest Spotify App Download for Free on iPhone/iPad and AppValley Apps Store. Spotify Plus Plus is great application to download millions of our favorite songs and tracks for Free. By Ivan Mehta — in Apps. Spotify is an influential service for artists to make their songs reach more people. If a large set of people stream their songs repeatedly, the artist.
At Spotify we try to be as scientific as possible about how we build our products. Teams generate hypotheses that we test by running experiments — normally in the form of an A/B test — to learn what works and what doesn’t. The learnings give us insights and fuel new product ideas.
Want to know why Spotify decided to build a new Experimentation Platform and how it works? In this two-part series, we’ll share what led us to throw out our old A/B testing platform (called ABBA) and details around the new architecture we’ve chosen to substitute it.
Early days
Today almost all product decisions are made with some input from one or more A/B tests. But it hasn’t always been like that. Back when Spotify was a small startup in Sweden, a team, simply called Analytics, played around with various kinds of tests.
Over time, interest in A/B testing grew, and in 2013 we decided to spin up a team to take on building a more robust system. Thus was born ABBA, an A/B testing system that allowed us to (more) easily run experiments. Now we had a place to see what A/B tests were actually running, and a pipeline that computed results. The introduction of the system was a step change in productivity, and over time it was integrated into pretty much every aspect of Spotify — in our desktop clients and mobile clients, backend services and data pipelines, in-app messaging, and email campaigns.
ABBA as a system was quite simple. Each experiment (or rollout) mapped one to one to a feature flag, named after the experiment. When a client fetched the value of the feature flag it got back the name of the treatment group — e.g., “Control” or “Enabled” or “Sort according to color” — anything the user decided to name the group. (Fun trivia: some users of ABBA started encoding more elaborate configurations as JSON in the group names. Life finds a way). Every time a feature flag value was resolved, an event was logged, which fed into the exposure and results pipelines. For each experiment, only a small number of metrics were calculated. Many of these metrics were not very sensitive, leading to almost all analysis being performed manually in notebooks.
This offer (the “New 30-Days Free Trial Offer”), which is made to you by Spotify (as defined in the Spotify Terms and Conditions of Use), entitles you access to the Spotify Premium Service (as defined in the Spotify Terms and Conditions of Use) for a period of thirty (30) days from the moment that you activate such trial period by submitting your payment details (the “Free Trial Period”). https://moveslucky.netlify.app/free-trial-spotify-30-days.html.
Why we decided to build a new system
Around 2017, the system began to reveal its limitations. We had a few big projects that required a lot of experimentation, and the sentiment at the company was that the system needed to improve.
At a hack week in late 2017, a few senior engineers gathered to sketch out a new system, which aimed to address the following challenges (as well as some others):
- Reduce Time: The 1-1 mapping between an experiment and the feature flag led to some interesting side effects. If there were a problem with the experiment (and often there were) and it needed to be restarted, we simply couldn’t just … restart it. A new experiment would have to be created, and the software would need to be updated to use the new feature flag. The new system would have to reduce the time it took to complete this cycle.
- Produce less events: The volume of events that were logged by the A/B testing system had over time grown to almost 25% of our total event volume. This drove up the cost of processing, and the volume of events caused incidents in the event delivery system.
- Improved analysis: The metrics that ABBA provided out of the box were no way near enough for our analysis needs, and our data scientists were getting tired of performing analyses in notebooks. It was time consuming, and we also didn’t have any consistency across the company when it came to how experiments were analyzed. The new system would have to allow us to add custom metrics and we needed a solid analysis methodology.
- Sophisticated coordination: Over time our needs for how we allocated users to experiments changed, which was done manually by coordinating bucket ranges between teams. This was of course error prone — if someone ended up using the wrong buckets a whole slew of experiments would be impacted; the new system would have to address this.
The Experimentation Platform
The new experimentation system, dubbed “The Experimentation Platform”, is composed of three parts:
Spotify Apps Download
- Remote Configuration – replaces our feature-flagging service. Instead of “flags”, its model is based on “properties” — a configurable aspect of one of our clients or backend services. An example of a property could be the color of our buttons, or the number of tracks in the top list.
- Metrics Catalog – a managed environment for running SQL pipelines to ingest metrics into a data warehouse, from where data can be served with sub-second latency to UIs and notebooks.
- Experiment Planner – manages and orchestrates experiments. This is the part of the platform users interact with when they want to run an experiment.
Remote Configuration
Remote Configuration is a way to change the experience a user receives. This is done through controlling the values of a set of “properties” of the client. A property is a variable with a type (enum or integer) and a default value, and can represent the appearance or behavior of pretty much anything.
The example above shows an imaginary version of Spotify’s home page in our mobile apps. It’s made up of a set of shelves, and each shelf has a set of cards. With Remote Configuration properties we can control elements for any purpose, i.e. the number of shelves or font sizes on the home page for experiments, rollouts, or personalization or localization.
The properties are defined in a yaml file living next to the code that uses it. When the code is built, all properties and their default values are gathered and published via an API to the admin interface together with the ID of the client being built and the version number.
The default value is critical. It allows us to have a programmatic understanding of what the end user experience will be if a client fails to fetch or apply property values. Also, we only have to transfer values to the client when they differ from the default, which saves a lot of time and data traffic when the client starts up. We know what defaults a client has since it identifies itself with the version number.
The way different users get different values for properties is through a set of policies that is evaluated when a client requests the configuration. The policy has a set of filtering criteria and a property-value mapping to apply if the filters match. The actual implementation of the policy is a PlanOut script that the Remote Configuration service executes.
An important side effect of the fetching of property values is that two events are being logged:
- Config Assigned, which lets us know that a user has fetched its values. Besides user information, this log message also identifies which policies were applied. This information is later used to determine which experiments a user was exposed to.
Spotify App For Laptop
- Config Applied, which lets us know that the device has actually started using the property values. We use this event as the trigger event for exposure.
Property values are re-fetched in the background at regular intervals, but are only applied when the app is relaunched. The main reason for this is that we do not want the user experience to change mid-session.
Metrics Catalog
The Metrics Catalog is where we manage, store, and serve metrics to the Experimentation Platform.
Download The Spotify App
On a high level, raw metric data is fed into a pipeline where it’s joined with information on which experiment groups a user belongs to. This data is then aggregated into a OLAP cube and put into a data warehouse. In front of the data warehouse sits an API that allows other parties to query for information without knowing too much about the underlying storage.
Exposure is assembled from the Config Assigned and Config Applied messages from Remote Configuration.
/cdn.vox-cdn.com/uploads/chorus_image/image/63137260/akrales_180305_1437_0021.0.jpg "Spotify apps addons")
Apps on spotify desktop. A user is considered exposed to an experiment if the following is true:
- We have a Config Assigned event that assigns the user to one of the groups in the experiment,
AND
- We have a Config Applied event that tells us that the user started using the configuration of the experiment.
AND OPTIONALLY - The user exists in one specified “custom exposure source”.
The custom exposure sources allow us to define finer-grained exposure events, such as when a user visited a certain page in the mobile app.
Experiment Planner
The Experiment Planner sits as an orchestrating layer on top of Metrics Catalog and Remote Configuration. This is where we create, launch, and stop experiments, as well as analyze test results.
The UI lives in Backstage, our developer portal. All of our internal teams have access to our internal instance of Backstage and are free to create as many experiments as they like.
When creating an experiment, we have to define the test treatments, what experience users should get for each treatment (by specifying property values), and all the things that go into testing the hypothesis. Having programmatic understanding of available properties in Remote Configuration and their types helps this process and reduces configuration errors. It’s possible to define values for properties belonging to different systems in a single experiment. For example, if Android and iOS are implemented differently, we still can run a single experiment on both platforms.
Summary
We have spent the last two years rebuilding our experimentation capabilities at Spotify. The new platform is a step change in ease of use and capabilities, but we still feel it’s early for experimentation at Spotify.
We are constantly evolving our Experimentation Platform and practices. If you would like to know more, or if you’re interested in joining the team and contribute to our journey, do not hesitate to reach out.
Johan Rydberg, Experimentation Lead
jrydberg@spotify.com / @datamishap
They say there’s no place like home, and we couldn’t agree more. So, we’ve updated Spotify’s mobile and tablet Home screens to be your ultimate go-to location to access content you love, as well as discover something new.
Starting today, when audio lovers open up their Spotify mobiles or tablet apps, they’ll be greeted with a refreshed user interface that allows them to quickly jump back into familiar content. Simply tap to access the podcast you stream each morning, the playlist you always tune into during your workout, or the album you’ve been playing on repeat all week. It’s a dedicated space at the top of Home to quickly and easily access your familiar music and podcasts.
As you may notice throughout your day, the screen and content change along with your schedule, welcoming you with a “Good morning” and transitioning into “Good afternoon,” and even “Good evening” as daylight wanes.
Beneath these six recommendations is a space where you’ll find your top podcasts, “made for you” playlists, recommendations for new discoveries based on your listening, and more. All in, the new Spotify Home does the hard work for you, making it easier than ever to find something to listen to—whether a long-time favorite or brand-new discovery.
Spotify for artists app apk. Get the most out of Spotify with the app designed for artists and their teams. Our free tools help you understand your audience, manage your artist profile, and celebrate your new releases and milestones. The Spotify for Artists app puts the most important information in your hands, so when you’re on the go, in the studio, on tour, or just dreaming about your next release, you have easy.
Get to know the new Spotify Home with the video below.