Super Collider
Empowering data-driven solutions through accessible data.
.png)
Overview
Super Collider is a scalable self-service big data analytics platform tailored to meet the diverse analytical needs of VMware’s business, operations, support, sales, marketing, and executive teams.
Role
Lead Designer - user research, end-to-end design process, aligning with business strategy, & implementation support.
Team
2 Product Managers, 1 Engineering lead, 1 Product Designer, and a Customer Success team.
Impact
Outstanding results, with a 90% reduction in users reaching out to the Super Collider team for data access and data inquiries, as users now correctly reach out to the respective data owners after implementing the new design.
What does Super Collider really do?
Super Collider is made to handle all kinds of data analysis for VMware, steering the organization toward data-driven decisions. It serves multiple purposes across VMware's business, operations, support, sales, marketing, and executive teams.
Let's illustrate one use case of Super Collider with a specific example of how the finance team at VMware utilizes it:
At VMware, we charge VMC customers based on the hours their AWS hosts were active. The finance team requires a report to track the running hours for each customer, enabling accurate invoicing. To generate this report, we rely on data from two sources.
The first source is product telemetry, providing billable hours for each host. However, this data only includes IDs without customer contact information. To link IDs with clients, we integrate data from Salesforce. In this process, our software engineers create two collectors—one for VMC billable hours with IDs and another for Salesforce data.
The VMC collector is meticulously set up, with the schema tested in staging before promotion to production. For Salesforce, a data engineer collects the necessary data and deploys it to Super Collider through a data job. Super Collider, functioning as a central hub, regularly ingests and combines this diverse data using data jobs and Data Marts. This structured approach ensures that teams maintain control over their data jobs, with visibility into deployment status and execution details.
Once the data is configured in Super Collider, our data analyst extracts it from the Data Mart, utilizing third-party reporting tools like Mode Analytics or Tableau to create a comprehensive report for accurate invoicing based on host uptime.
First Things First: Domain Research
I joined the team in 2019 with a mandate to improve our products for our internal users, to help internal teams in VMware be able to get data easily, and to empower data-driven decisions through the company.
As a UX designer crafting interfaces for intricate applications in specialized domains, a deep dive into these domains is essential. Understanding the platform's functionality, user base, and key user flows is crucial. In this role, I delved into data analytics terminology such as "data marts", "schema," and "data jobs," unraveling their interconnectedness. To enhance user experience, I familiarized myself with the roles of data engineers and data analysts, comprehending their end goals within the Super Collider framework. This knowledge allowed me to grasp the underlying logic of workflows and intricacies of the data analytics realm, empowering me to better assist our users and enhance their overall experience.
.png)
Starting with the high-level workflow first.
.png)
Drilling down into the key workflows our main personas perform.
I started with stakeholder research. We had a customer success team, which was mainly built of data engineers and data analysts. This was an amazing opportunity to get domain knowledge from people who represent our end users as well. I had sessions with product managers, engineering leads, data engineers, and data analysts, walking me through the main user flows and explained what the key workflows are and what the terminology behind them means.
.png)
In a meeting with a Customer Success data engineer: identifying and illustrating he steps taken by both data engineers and analysts in their tasks
Uncovering UX Debt: Heuristic Evaluations & Expert Review
As I onboarded and familiarized myself with our domain and user flows, I identified several areas for improving the user interface.
To address these concerns, I established a Product Design Confluence page dedicated to newly discovered UX Debt. Using the "Expert Review" method, I assessed the design for compliance with heuristics and other usability guidelines, drawing on cognitive psychology and human-computer interaction principles. The expert review format included a screenshot or video of the usability issue, a problem description, severity level, recommendations, a link to Jira for tracking, and additional comments if necessary.
In total, I identified 37 usability issues with varying severity levels. The next step involves adding them to Jira and prioritizing them collaboratively with the team.
.png)
Conducting "Expert Review" and capturing the Super Collider's usability issues in Confluence.
Focus on User Research: Contextual Inquiry
As my grasp of the world of data analytics and the main 'jobs to be done' by our users deepened, the next step was conducting contextual inquiry with actual users.
We wanted to better understand our users’ current workflows and pain points to provide better experiences for the most important use cases. For this, we decided to conduct a contextual inquiry with a typical in-situ interview, followed by a period of observation and questioning. This allowed us to see where our users struggle when going through the key use cases.
I created a research plan and a research protocol and then conducted Contextual inquiry with 12 users (5 software engineers,5 data engineers, and 2 data analysts.)
Outcomes: Discovered usability issues and defined user personas
Case Study: Data Access
One key concept of Super Collider is the flexibility for teams within the company to utilize their teammates' data for their reports as required. Teams can decide whether to keep their data public or private. In the case of private data, one can request access to the team's data, and only after obtaining approval can it be used. Notably, about 70% of the data in SC is stored in private marts, making the request for access to private data marts a frequently encountered scenario.
The Problem
A key insight from our research reveals that data engineers and analysts often assume that the Super Collider team owns the datasets in Super Collider, which is not the case. There are three issues stemming from the assumption that the Super Collider team owns the datasets in Super Collider:
-
Firstly, users face challenges in accessing private datasets due to a lack of awareness that individual teams are the actual data owners. This confusion leads to frustration as users instinctively seek access from the Super Collider team instead of the relevant team.
-
Secondly, when datasets are made public, users reach out to the Super Collider team with questions about the data, assuming the Super Collider team is the proprietor. This assumption, even with public datasets, adds complexity.
-
Thirdly, these dual challenges not only complicate user access but also overwhelm the customer success team with requests and inquiries, despite their lack of control over the data.
There are two user interfaces that the users can browse through the datasets: the Super Collider UI and the Super Collider SQL UI.
In the Super Collider UI, the visual cues are very low on whom to contact when you want to request access or get more information about the data. This results in users reaching out to the Super Collider team for access.
Team information on the left side of the card illustrating the data owner's team was often overlooked, leading users to contact the SC team.
In the SQL UI when a user queries a dataset that they do not have access to, they get an error message that states that the database does not exist or access is denied. There is no guidance on what the user needs to do next or who they need to reach out to for help. In addition to that, when users have access to and view datasets, they often have questions regarding that data and they still need to contact the owners of the datasets. As a result, in both cases, 10 out of 10 times the users reach out to the Super Collider team, assuming the SC team owns the data, instead of the data owners themselves.
.png)
Users are stuck when they see this error messagem, as it lacks information of what they need to do next.
Brainstorming and Ideation
To request access or pose questions about the data, Super Collider users should reach out to the team that owns the data. In the current Super Collider user interface (UI), users can accomplish this by navigating to a specific dataset, clicking on the "Send Email" button located on the right-hand card, and composing an email addressed to the respective data team owner. In SQL SC UI, there is no way to understand who to contact when you have questions about the data or you need access to it.
In a team brainstorming session, numerous ideas surfaced for enhancing the situation. We discussed improvements centered around optimizing the process, such as transitioning from email requests to implementing a new system where responsible team members could efficiently approve or decline requests. However, we had to remind ourselves that our primary goal was to identify a low-effort, high-impact solution, with a key success metric being awareness among Super Collider users.
Success, in this context, means that users are cognizant of the need to contact the respective teams rather than the SC team, ensuring that requests and questions are directed to the appropriate recipients from the start.
What we tested
We had an assumption that the reason why users kept on contacting the Super Collider team is because of the poor visual cues in the Super Collider UI design and the poor call-to-action. When it comes to the SQL UI, the reason was the lack of any feedback.
Super Collider UI: We tested different design tweaks using the Clarity design system. We aimed to identify elements that effectively capture user attention without inducing a sense of urgency.
SQL UI: The SQL UI did not know who owns the different data sets, as this knowledge was in the Super Collider UI only. This meant that at this time we could not use a mailto link in the SQL UI, but regardless of that we still had to give feedback to the user.
.png)
Different design exploration that we tested with users.
Small Changes - Significant Impact
In search of a low-effort, high-impact solution, we emphasized the call-to-action—prompting users to contact the data owner team for access and information. The new design amplifies attention by incorporating the secondary call-to-action button from the Clarity design system. Additionally, I revised the card header from "Contact Data Owners" to "How Do I Request Access and Use the Data?" creating a more compelling focal point for users seeking answers.
In the SQL UI, user feedback now explains the lack of access and outlines the steps to obtain it. For data information within datasets, we introduced a constant help button, ensuring users are consistently reminded of the current approach to obtaining information and how to contact for assistance.
After launching this, the results were outstanding, with an impressive 90% reduction in users reaching out to the team for data access and data inquiries.
Emphasized visual and text cues
Enhanced visual cues to direct our users and immediately raise awareness about who they need to contact for data access and data inquiries.
.png)
Clear feedback with next steps
.png)
Clear feedback with next steps
Recognition
I have received an award recognition from the Super Collider Product Manager for my contribution.
Iliyana,
Thank you for your hard work, determination and the ability to not give up the best UX practices, even when we made it really hard for you with our non-UX-background suggestions! I really appreciate it!
Many thanks,
Ivan