This repository contains the frontend UI code for Colocus. Users will likely want to view that repository instead for information on how to deploy the app on their own server. Below is primarily documentation meant for frontend developers.
The frontend is a Vue 3 app built using Vuetify 3. My overall philosophy is to make every file as concise and focused as possible, trying to limit line count to 100 or so. In many cases, this entails moving functional code to one or more composables, such that the primary view has two jobs: rendering templates and functioning as traffic cop for event handling. There are a few files where line count greatly exceeds that target, because all the functionality was logically related. For example, the mzGridHelpers composable tops out at over 500 lines.
I have tried to avoid use of magic numbers and strings in code, and centralize constants in the file src/constants.js. This goal has slipped a little in the final months, so there are a number in the Multizoom page and related files.
Overall page structure is defined in App.vue, which contains the toolbar, a main section, where router views are displayed, followed by the footer. Each page of the application pages is defined in a separate view file in the src/views folder. Each view is defined by a layout (see src/layouts for examples). The DefaultLayout is for 'whole-page' views, such as home, help, studies. The SidebarLayout is used by most functional pages, which have a sidebar known as the FilterPanel, plus a main content area for displaying data tables and plots.
- Prefix event handlers with 'on', eg onRowClick
- Prefix booleans with is, eg: isLoading
- Put destructured imports in alphabetical order, eg import { onMounted, provide, ref, watch } from 'vue'
- Imports should be in this order: Vue, third-party libraries, app-level imports
- List variables, props, watches, computeds, etc., within a section, in alphabetical order
I find that it helps reduce cognitive load to have the sections in the same order throughout the application. There are some dependencies among them; for example, variables must be defined before watches. Very simple components don't need the overhead. This structure does not fit composables, constants, and helper files. In those cases, functions are listed alphabetically. Within the delimited sections, elements are listed alphabetically.
// *** Imports *****************************************************************
// *** Composables *************************************************************
// *** Props *******************************************************************
// *** Variables ***************************************************************
// *** Computed ****************************************************************
// *** Provides ****************************************************************
// *** Injects *****************************************************************
// *** Emits *******************************************************************
// *** Watches *****************************************************************
// *** Lifecycle hooks *********************************************************
// *** Event handlers **********************************************************
// *** Utility functions *******************************************************
// *** Configuration data ******************************************************
The component is a composite that is defined by a set of nested files in its own folder under src/components. It consists of several subpanels, each of which contains a set of controls. Key elements of the component include the following:
- a template section that defines the component hierarchy.
- a data structure, controlConfig, that contains configuration data for all of the controls
- regex patterns used for input validation
- rule functions used for input validation
Standard Vue props pass key data elements down to the subpanels and individual controls.
Each subpanel has an optional reset button, which will reset its contained controls back to a default state, which is passed to the controls via the controlSet prop, as specified in the top level . The subpanels and controls utilize Vue 3's provide/inject mechanism to pass values through several layers of components. For example, when the user clicks the Reset button, it flips the value of a local ref variable, resetInput, and that variable is used by the provide mechanism to pass down to the actual controls.
Then, in the underlying controls, Vue's inject mechanism allows monitoring of that variable's state. When the state changes, the control reloads itself, and updates the Pinia store with the default value. For example, all the child components of a given filter sub panel respond to the same reset signal.
All of the underlying controls are composites, consisting of a text label, whose value is passed in as a prop (controlSet.title), plus a wrapper around a standard Vuetify control (AutoComplete, TextField, Switch).
The component also listens for changes to a second injected signal, loadFPControls. This is provided by the top level pages (Search and LocusZoom), and is toggled in their onMounted events, when it is safe to assume that the search metadata API call has returned and stored its results in the Pinia store. So when a sees that flag toggled, it loads the select list with the appropriate data from Pinia. The control also loads its own (user) data in the onMounted event. This is needed when transitioning between the Search and LocusZoom pages, and other pages. The function populateControlData handles this, including a mapping from the current page name, so that it knows which data element in the Pinia store to access. Last, a modelChanged function updates the store upon user input.
The component is similar, but simpler. It also listens for a reset signal, and responds to user input by updating the Pinia store.
The component is similar. It also listens for a reset signal, and responds to user input by updating the Pinia store. It also debounces user input, to avoid needlessly triggering API calls upon incomplete input. It also applies the passed-in rule to the user input. On invalid input, a message, which is actually part of the rule definition in is displayed in red by the control. The only other thing of note is a function, trapEmpty, that checks whether the control has a value when it loses focus, and if not, provides an 'empty' value, passed in from the data structure in the parent .
There is a Vue 3 composable, featchData.js, that handles data loading. Getting it working with the filter controls was tricky, so worthy of some explanation. Initially I tried loading the search metadata in the App.vue file, but it actually appers to load its elements in an async manner. So, for example, you can't assume that the Pinia store is ready when loading a page such as Search. Normally it would be, as you would access it after loading the Home page. But if you force-reload on the Search page, the page will display before the store is ready. This would also occur if a user bookmarked the search page (even though at present their filter settings would not be restored on direct access to the page).
The most reliable way I found to load the search metadata was to use the route guard, beforeEach, which loads the filter data if it hasn't already been loaded.
The component, along with all child components, is defined in a subdirectory of /src/components. In order to reduce the complexity of the main file, some routines and configuration data have been moved to a composable, DataTableHelpers, in /src/composables.
The way it currently works:
Search page loads. It has provide flags for:
- loadFPControls
- preLoadGenes
- loadDataTableFlag In onMounted, loadFPControls is flipped to load the filter control lists. Then either preLoadGenes or loadDataTableFlag is flipped, depending on whether there is a gene specified in the url.
The Manhattan page has the same flags.
The DataTable component injects the loadDataTableFlag and watches it. Whenever it changes, the watcher executes loadTableData.
Whenever filter data changes, the associated control calls appStore.updateFilter. It then flips filterDataChanged. The DataTable watches that, and executes loadTableData whenever it flips.
Whenever the user changes page or page size, appStore.updateFilter is again called, which flips flag filterDataChanged. There is logic as to when it will do so. Mainly, we want to ignore a page num event immediately after a page size event. Otherwise, the flag if flipped twice in succession, and the watch in the DataTable misses the change, so data is not loaded.
Currently, there are three Pinia store instances. The appStore is used across most if not all pages of the app. The other stores have more focused applicability.
The appStore holds all app state. At the top level, this includes a set of general keys, plus a set of keys for the main views (search, locuszoom, manhattan (probably, assuming it needs filter)). A function provides the data structure to ensure consistency across pages and to avoid repitition. There are also two objects that function as maps between 1) data key names in the app vs those expected by the API, and 2) sort keys expected by the back end.
This store started off supporting data structures required by the filtering system, but has grown, and may be renamed or split into separate stores as development proceeds.
The data structure includes global state variables (e.g., isDataLoaded), plus page-specific data for each page that uses a filter panel. The page-specific data includes the values of the controls, plus additional entries for page size and page number, all of which are brought together, along with the sort keys, to build the URL for the Django back end.
- buildSearchURL builds a URL based on all filter and sort criteria and formats it for consumption by the Django REST API back end.
- checkGenes accepts a comma-delimited list of genes and returns an object containt two arrays, badGenes, and goodGenes. This is used in the case where an external entity (such as AMP) specifies a gene as a query string in a URL directed at the search page.
- copySearchFilterToLZ copies the filter panel data for the search page into the corresponding keys for the LocusZoom page, and is called by the router before entering the LocusZoom page.
- loadFilterData relies on the async composable, fetchData.js to load the filter data lists and populates the filterPanelControls keys for use by the FilterPanel controls.
- updateFilter is a helper for the filter panel components. updateFilter updates the appropriate value in the appropriate key, depending on which page (Search or LocusZoom) the user is on.
- updateSort updates the store key storing sort parameters that end up in the URL.
- updateSwitch updates the booleans tracking the display of the ensemble IDs and the concordance values.
- toggleSidebar is used by the to show and hide the filter panel.
For most pages, the router simply provides a link to the corresponding view file. The Search and LocusZoom pages enable the filter panel and button, and the LocusZoom page copies user filter panel selections and entries from the Search page data.
There is a global route guard, beforeEach, which disables the filter panel and loads filter data if needed.
A key design goal of this iteration of Colocus was to centralize style definitions as much as possible, colors in particular. Our global constants.js file has a COLORS object that is the definitive source of colors throughout the app. Each color is prefixed with CLC_, which stands for Colocus Color. Then a functional designation is appended. For example, CLC_ACTION refers to our action color, used to style components where user interaction is possible.
Vuetify has an extensive color management system, based on Google's Material Design. Vuetify allows definition of custom themes. I have defined a custom theme in plugins/vuetify.js. Its colors are mapped directly from the COLORS object.
However, Vuetify's themes only cover Vuetify components, not native HTML elements, such as headings, spans, etc. The solution is a css file, styles/global.css, that references variables from the custom Vuetify theme. The global css file is imported in main.js. Following is an example of how this works from global.css.
h1 {
font-size: 2rem;
color: rgba(var(--v-theme-CLC_HEADING), 1.0);
}The details will likely change, but this illustrates the principle. The rgba(var(--v-theme-), a.b) expression specifies the color name defined in the Vuetify file, with an optional opacity value of a.b, which may range from 0.0 to 1.0, so tints would be possible.
The file src/ide-helper.css functions to prevent spurious warnings from the WebStorm IDE about these var expressions. The file is not imported or otherwise used anywhere in the project. Each custom color added to the custom theme in vuetify.js should have a corresponding entry here. The value shouldn't matter, as, again, the file is not used by the app. Without this file, the IDE presents a distracting warning wherever a var expression with a --v-theme- is used.
The following URL provides a tool useful working with color definitions. For example, you can give it a hex color, then select different shades and tints based on it. https://www.w3schools.com/colors/colors_picker.asp?color=18c11c
It is unnecessary to explicitly import components in this app. Importing is handled by a plug-in called unplugin-vue-components. This plugin automatically imports .vue files created in the src/components directory, and registers them as global components. Then, when the template is rendered, the appropriate import statement is injected.
This means that you can use any component in the application without having to manually import it. You can add additional folders in the plugins.Components.dirs array in vite.config.mjs. The advantage is that it simplifies refactoring the src/components directory. The WebStorm IDE currently does not understand this, so it highlights non-imported component references as if they were errors. Note: If you move files around while the app is running, you'll have to restart the node server.
The Search View provides a context for displaying a data table. It uses the Vue 'provide' mechanism for three variables.
- loadFPControls: a Boolean that when toggled tells the underlying controls on the filter panels to load their select lists (from the Pinia store)
- loadTableDataFlag: a Boolean that when toggled tells the data table to load data.
- preloadGenes: an array variable that when set tells the underlying Genes control to adopt the provided values as if they had been selected by the user. (The values actually come in through a URL.) Setting this value also triggers a data load of the main data table. The sequence here is:
- Extract gene string from URL query string
- If there are valid genes, update preLoadGenes
- Through provide/inject this causes the underlying AutoComplete to call the updateFilter method on the Pinia store
- UpdateFilter adds the selected gene(s) to the filter data, then toggles local variable filterDataChange
- The DataTable component watches the variable and triggers a data load when it changes. The reason the DataTable has to watch appStore.filterDataChanged, instead of using provide/inject, is that it is not possible to use that mechanism from within a Pinia store. The effect is the same, it's just a different way of triggering a desired event.
This is a feature added and then disabled. It was originally added due to hope for the Manhattan view to display data in the data table from multiple studies, and allow the user to filter by phenotype. But then it turned out that the back end as presently configured, could not support this. I am disabling the code, but documenting here, in case we decide to go back and correctly implement that feature.
On the Search page, the user clicks a trait link. The router pulls trait from the URL and saves it in the appStore as a top-level property called preloadTrait. The Manhattan view copies that to a local variable in its onMounted lifecycle method. It provides that value for injection where needed, in this case the AutoComplete control. It watches that variable, and when it is updated, assigns it to local variable selectedItems, thus populating the on-screen control, and then calls updateFilter on the filter store with that value, which causes the data table to load.
The Search page also has references to preloadTrait, which are not used, but are required so that the AutoComplete controls will function.
To re-enable the feature, search across files for preloadTrait and remove the comment markers. Also have to add the trait name to the router-link passed by the data table. And add :trait to the path in the route to the Manhattan page in the router.
The help page exists in source code as a markdown file, help.md. It is rendered into HTML by the marked library. The rendering is controlled by Vite, in the vite.config.mjs file. In it is a function, markdownPlugin, which is registered as a plugin in defineConfig. The function is called whenever the underlying markdown file is changed, or when Vite builds for production.
The file HelpView.vue defines two columns, the left for a table contents, the right for the actual content. Clicking a link in the TOC causes the right side to scroll to the associated position in the document.
Note that although the markdown file is referenced, it is the HTML file that is actually rendered.
The TOC must be maintained manually. Its href values are determined by the render function in the vite config file.
A Vue watch is set up to watch the v-sheet that contains the help content. When it is populated, the watcher sets up the event listeners for the links, and when it is being torn down, the event listeners are removed. This is necessary to prevent memory leaks, and to avoid navigation problems with the Vue router.
This is the most complicated page of the app thus far. It consists of five main files, plus references to the appStore, utils, etc. The main files are:
- LocusZoomView.vue: This is the top-level page definition linked to by the router. It primarily consists of the Vue template definition, plus event handlers.
- lzPageHelpers.js: This is a composable that contains the meat of the functionality required by the page.
- LzPlot.vue: This is a wrapper, lightly modified from the Vue 2 app, that allows creation of the main containers for the compare and region plots.
- lz-layouts.js: This is a configuration file, more heavily modified from the Vue 2 app, to provide more flexibility in creation containers and plots.
- LDPanel.vue: This ia Vue single-file component, the provides the UI controls for operating the page.
Our common idiom for data loading in this app is to set a flag from the source requesting the data, then watch that flag and load data when it changes. That pattern is used in the LZ page as well. The onMounted lifecycle hook initiates data loading.
First we set the appStore.locuszoom.tableDataLoaded to false. Later, when the data table's loadData is called, this alerts it that the data is needed.
Then we flip the value of local variable, loadFPControls. This is provided to controls in the filter panel to alert them to load their static data, such as the gene list, not user selections.
Next we set the flag appStore.colocDataReady to false. The data table's loadData function sets that flag to true when the coloc data is ready. Then we have several watchers that take appropriate action when the colocalization data is ready:
- the LDPanel component, which allows to set the correct radio button for the initial LD reference
- the LocusZoom page, and this is what kicks off loading of the compare and region plots.
This version of the code programmatically creates the compare container and plot, plus the region container and plots, including the gene panel. To accomplish this, we removed the static configuration data from the definitions in lz-layouts. Most of it was discarded, but we needed to use two static definitions (gene_selector_menu, and genes_layer_filtered), which we embedded in the LZPLot.vue file. We replaced the static definitions of the region plot and the gene panel with a programmatic approach. See the functions addRegionPanel() and addGenePanel in LZPLot for details.
We also refactored almost everything except the template definitions and event handlers out of the main page view (LocusZoomView.vue), and into the composable lzPageHelpers.js.
Placeholders for the compare plot and the region plot are defined in the template as empty divs:
...
<div ref="comparePlot"></div>
...
<div ref="regionPlot" class="region-plot"></div>
...
Then the ref variables are populated by functions buildCompareLayout and buildRegionLayout in the composable. Internally, each uses an advanced Vue feature called VNodes. Vnodes are virtual nodes that Vue uses to track the entire structure of an application. At the highest level, declarative templates are compiled to Vnodes and then assembled into a virtual DOM. Vue tracks changes to the virtual DOM and periodically transfers changes to the actual DOM. Our build Layout functions create Vnodes based on the underlying LZPlot library. The *VnodeRef variables maintain Vue references to these components so that we can do things with them later, such as adding additional plots.
A key function in the composable, assembleLayout(), builds the overall containers (compare and region), then adds the initial plots to the region panel.
buildRegionLayout(), which, replaces the old static configuration, builds the container that will hold the region plots, saves a ref to the Vnode, and populates the regionPlotRef, which is passed in from the LZ page, and is thus displayed by Vue's reactivity system.
Similarly, buildCompareLayout(), builds the container, and also creates the scatter plot directly.
The LDPanel is displays operational controls for the LZ page, including a list of variants to be used as LD references in the region plots. The list is generated in the composable and then pushed to the appStore. Earlier, I tried to supply the list directly to the LDPanel, but it would not display correctly. In different scenarios, it would display only the first two elements, or duplicate the first two elements as panels were added, or display nothing in the label slot. The functional workaround was to push the list to the appStore.
So LDPanel pulls the list from there for display as radio buttons. Using our VariantLabel component caused erroneous behavior. The solution was to render and format the values directly.
- use {{ $log() }} in templates to log local values to console. This is defined in main.js.
The following may be copied in part or wholesale and pasted into the gene control in the filter panel
A2M,AAMP,PDF A2M,AAMP,PDF, A2M,AAMP,PDF,x A2M,AAMP,\t\r\n\n PDF,y A2M,A2ML1-AS1,AAGAB,AAK1,AAMP,ABCA1,ABCA8,ABCG5,ABCG8,ABHD12,ABLIM3,ABTB1,ENSG00000000938,x,y A2M A2ML1-AS1 AAGAB AAK1 AAMP ABCA1 ABCA8 ABCG5 ABCG8 ABHD12 ABLIM3 ABTB1 ENSG00000000938 x y
z vv x1
In the fall of 2024, we implemented the front-end testing framework, Cypress, learned a bit about it, and implemented some basic tests. The result exists in branch front-end-testing-with-cypress. It was built on Cypress 13.16.0. It includes a single file (test1.cy.js) with our tests, plus two folders of sample tests supplied by the vendor for reference.
To run cypress, open an additional terminal window in the project directory. The node process running the front end as well as the Django process both need to be running. Then run cypress:
$ npm run cypress:open
This executes a script defined in package.json. It will open a window, which is the cypress app. It presents two buttons, E2E Testing and Component Testing. We did not implement any component tests, so click E2E to get started. Next a browser selection window opens, and will show browsers installed on your system. We only testing using Chrome. Select Chrome if not already selected, then click the green button, Start E2E Testing in Chrome.
A new special Chrome instance will open full screen. Resize as desired, then scroll down to the bottom of the list to find test1.cy.js and click it. It will run all tests in the file and all should pass. (Actually with code as committed, will only run the study selector test. Remove the .only tag on it to run all tests.)
As this was a proof-of-concept, there are several unrelated tests in the file. Each group of tests is contained in a describe block. Each block is defined by a text string that is output when tests are run, and further serve as documentation, plus a callback function that contains several tests. Each test is defined by an it block, again with a text string and a callback containing a single test.
The first describe group validates basic navigation to the currently active pages of the app. The second tests some functionality of the search page - mainly, the filter panel.
Cypress supports the concept of hooks. In our case, both describe blocks use a beforeEach hook, which is run before each test. For the navigation block, the beforeEach simply loads the home page via the visit command, and for the search page, it loads the search page.
Note the test, "test the study selector". It is defined by it.only, which means only that test will run when the file is executed. The first line of the test enters (types) the desired study and forces it to be selected.
cy.get('#studyInput').type('AdipoExpress{enter}{esc}{esc}')
The remaining lines then execute different versions of the same test, which insures that the first row, third column of the data table contains the string 'Adi...xpress'. The first two of those are commented out, as they are undesirable from the standpoint of the selector used to find the component of interest in the DOM.
The first commented out test drills all the way from the vue #app component.
The second drills from the component #dataTableSearch (which is actually a div containing the actual HTML table). One element in the select chain is [data-v-dd96f50a=""] which is an attribute on a span created by Vuetify. I was concerned that the dd96f50a might change between installations or versions of Vuetify, so looked for a more generic approach.
The third also drills from #dataTableSearch, but then uses the Cypress within command to penetrate to the lowest level span containing the desired text string. Within takes a callback function; within that we perform another get command to drill through the nested spans and get our test string.
There are actually sub-versions of that; one contains the literal string, the other a regex. I was concerned that we might change the parameters of the shortening function that removes the middle of long strings, so the regex version is a little more general-purpose, but still probably not ideal.
You might wonder if simply replacing [data-v-dd96f50a=""] with span in the second test would suffice, and for unknown reasons it in fact does not work, hence the third version using within.
Since this was a proof of concept and we just wanted to get something working, we made no attempt to organize the tests. For example, the first block, navigation, would become a separate file. The second block is really testing the filter panel, not the search page, so it's description would be altered, and it would move to its own file.
The example below adds a placeholder page, QC, to the app.
- Add page name to PAGE_NAMES in ./constants
QC: 'qc',
- Add vue file to ./src/views
<template>
<h1>Bonjour, le monde</h1>
</template>
- Add entries to ./router/index.js
...
const qcPage = PAGE_NAMES.QC
...
{
path: `/${qcPage}`,
name: qcPage,
component: () => import('@/views/QC.vue'),
beforeEnter: (to, from, next) => {
next()
}
},
- Optionally, add link to ./src/components/misc widgets/Toolbar.vue if not otherwise linked.
<router-link to="qc" class="nav-link text-CLC_HEADING">QC</router-link>
The plotting system depends on the following components
- The overall vue pages (QCStats.vue, SummaryStatsView.vue)
- A vega spec for each plot, stored in src/vegaSpecs
- A Pinia store, QCStore, for plot data and operation
- A plot component, , one for each plot rendered on a page
- VegaPlotConfig, a js file containing configuration info for instantiating all plots
- qcMake*, a set of composables providing data processing functions for building the data sets required by the plots. Some of the simpler data sets are generated directly within the QCStore.
These views contain the overall page structure, including the templates for all the plots. Each page has two columns, one for the sidebar and one for the plots. The first row in the plots column includes the page header and description.
This is then followed by the plot templates, each an instance of . That component accepts two props:
- a controlSet, following conventions established elsewhere in the app, which has configuration values for each plot
- a Vega-lite spec, the details of each plot, including the data In this app, the spec is a JS object, not a JSON blob, which simplifies dynamically updating things like titles, colors, data set, etc.
The page's only action, in the onMounted hook, is to tell the qcStore to load the plot data from the back end, then flip the flag that triggers plot generation.
Manipulating the controls in the sidebar also causes plots to be regenerated on each change to the selected:
- study
- h4 threshold
- r2 threshold Pressing the 'reset' button to the left of the heading ('Settings') causes all three values to be reset to their default values. Resetting also forces the page to scroll to the top. Similarly, selecting a new study forces the h4 and R2 values to their default values, and a scroll to the top of the page.
The Pinia store for this portion of Colocus is responsible for fetching data from the API, and generating data subsets for the different plots. It has state variables for those data elements, plus variables for the UI elements controlling the plot displays (h4, r2, and omics study). The qcStore has three primary actions:
- loadQCData: fetches fundamental data sets from the backend and generates subsets for the plots
- makeRecordsForAllPlots: regenerates plot data when needed (on initial load and when UI changes occur). This uses several internal methods, as well as the larger composable functions.
- regeneratePlots: Flips the regenPlotFlag, which is watched by each of the plot instances, so when the flag changes, a new plot is generated.
- updateQCStoreKey: updates state variables when UI controls change due to user interaction
There are also several internal functions (getColocDataForStudy, getQTLStudies) used for data processing.
This component provides the template and run-time code to instantiate plots. The component is triggered by watching a flag in the qcStore (regenPlotFlag), and when that flag changes, the component builds a new plot instance and embeds it in the containing div in the template. The flag is triggered on initial load of each of the stats page views, and on subsequent changes to the UI controls.
Early development of this functionality went smoothly, but at a certain point, the time to render plots became excessive. Initially it was nearly instantaneous, but then grew to as much as 30 seconds in some scenarios. Eventually it was determined that the slowdown only occurs in dev mode. Running in production mode, plot generation and regeneration is instantaneous, following data acquistion.
Rather than having to deploy to the external platform, a server that can run locally was develoed. It uses the ExpressJS library server. It requires the project to be built, then serves files from the /dist folder. Details are in comments in /express-server.js. This file is part of the project, but its dependencies are dev only, so will not be part of a production bundle.
It is possible to run vite build --watch and have the dist folder updated in real time as files are saved. I added two npm scripts: one to build the project, the other to serve the contents of the dist folder. Both need to be running in separate terminal windows in order to have an automated edit-save-build-serve development cycle.
- dev-build: this runs vite build in watch mode, regenerating the site after each save
- dev-serve: runs the express server as a standalone instance
This works, but is a little slow; it takes about 4 seconds to rebuild the project. More significantly, Vue Dev Tools don't work in production mode. I recommend using this approach only for working on the QC page; for other pages, use npm run dev.
I attempted to optimize plot generation. Vega allows plots to be updated by supplying new data. The hope was that it would speed up redraws. However, it did not speed up the process, and furthermore left artifacts on the plot. I was unable to resolve either issue, so reverted to just regenerating the plot from scratch on each update of the UI controls. For potential future use, here is the code from VegaPlotContainer to generate a plot on initial use, then update it subsequently.
import embed from 'vega-embed'
import * as vega from 'vega'
// ...
const chartView = ref(null)
// ...
watch(() => qcStore.regenPlotFlag, async (newVal, oldVal) => {
if(!chartView.value) { // initial hit, create plot
qcStore.makeColocClassPlotRecords()
vegaSpec.data.name = controlSet.dataName
vegaSpec.data.values = qcStore.recordsColocClass
const containerID = `#${controlSet.containerID}`
const { view } = await embed(containerID, vegaSpec)
chartView.value = view
} else { // just update
timeLog('starting to make plot records')
qcStore.makeColocClassPlotRecords()
timeLog('made plot records, starting vega update')
chartView.value
.change(
controlSet.dataName,
vega
.changeset()
.remove(() => true)
.insert(qcStore.recordsColocClass)
).run()
chartView.value.resize().run()
timeLog('vega update complete')
}
})
// this also necessitates providing a name for the dataset in the vega spec:
"data": {
"name": "placeholder",
"values": []
},
// and in the VegaPlotConfig for each plot
...
dataName: "colocClassData",
...
Another performance issue: The UI became nonresponsive for long periods of time -- up to 15 mintues -- when plot data was stored as reactive data in the QCStore, so care must be taken to ensure that data is stored nonreactively.
The plot specs in the Observable file use a method of setting global defaults for overall plot height and width.
"config": {
"view": {
"continuousHeight": 600,
"continuousWidth": 600
}
}
Consequently, early graphs with smaller specified sizes adopted the sizes specified by later graphs. For example, Class of Colocalizations had a specified width of 600 but were actually about 800 wide.
So the config.view.continuous* were removed to set dimensions directly. In particular, setting only the width, and allowing Vega to set height itself produced better looking plots. Also, I removed the hard-coded value for width, and specified "container", thus allowing height to be specified externally by the Vue infrastructure. A single width setting is required for single plots, but multi-plot plots have to set the width of each element of the vconcat array.
Another issue is that Vega emits a warning, "VegaPlotContainer.vue:40 WARN Dropping "fit-y" because spec has discrete height." Attempts to resolve this were unsuccessful, primarily through setting the autosize property in the spec, as follows. There are currently other unresolved warnings.
// "autosize": {
// "type": "fit"
// },
// "autosize": {
// "type": "fit-x",
// "contains": "padding",
// },
Several of the plots (7, 8) require dynamic updates to the specs. For example, changes in the study drop-down require the titles and axis labels to be updated. In VegaPlotConfig.js, you can specify a key, axisTitles, with a title string template. At render time, will update any title(s) specified. Below is an example for Plot 7, Count of QTL signals colocalized per GWAS. Note that outermost keys (e.g., xTopTitle) are not used for anything, but serve to document the function of the given title, in this case the x axis of the top plot element.
axisTitles: {
xTopTitle: { key: "vconcat[0].layer[0]", value: "Count of %s QTL signals" },
xBottomTitle: { key: "vconcat[1].layer[0]", value: "Count of %s QTL signals" },
},
This data structure is processed in , where a lodash set function drills through the layers to update the value of the specified key.
A similar approach is taken to specify plot colors:
barColors: {
topTotal: { key: "vconcat[0]layer[0].mark.color", value: BAR_COLORS.BAR_TERTIARY },
topCount: { key: "vconcat[0]layer[1].mark.color", value: BAR_COLORS.BAR_PRIMARY },
bottomCount: { key: "vconcat[1]layer[0].mark.color", value: [BAR_COLORS.BAR_PRIMARY] },
},
Instead of recreating the whole plot each time the UI changes, in theory we could force a plot update. In that case, the following key would be added to each config element. However, couldn't get update to function reliably.
dataName: "colocClassData",
A similar approach is used to set font sizes of
- y-axis lablels
- overall title
- bar mark text
- legend text, if any
- if needed, add data set to qcStore, make sure nonreactive
- add record to VegaPlotConfig.js, setting values appropriately
- create spec and add import to QC.vue
- add row for new plot, setting controSet and vegaSpec in the tag
Observable name Colocus name plot Description
----------------------- ------------------- ------- ----------------------------------------------------
allColoc colocAll all from api/v1/internal/coloc-slim/
records colocClass plot1 graph built from coloc, used for coloc class plot
colocForStudy colocForStTi allColoc matching study, tissue
recordsWithoutOneToOne colocWithout11 plot2 records excluding oneToOneSignal
colocForStudyWithH4 colocWithStTiH4 allColoc matching study, tissue, h4
coloc colocWithStTiH4R2 plot3-6 allColoc matching study, tissue, h4, r2
countsForFigures["byOmics"] contsByOmics plot7-8 data derived from signalsAll
countsForFigures["byGWAS"] contsByGWAS plot9-10 data derived from signalsAll
signalsPerDataset signalsPerDataset plot11 data derived from signalsAll
allSignals signalsAll all from api/v1/internal/signals-slim/
qtlStudies qtlStudies map of all studies from allColoc
countsForFigures ? filtered allSignals, uses colocForStudyWithH4
Top Section, gray ALL GWAS bar
- barLayerTopTotal: shows gray bar
- barLayer: overlays purple bar for sub sum
- textLayerTotalTop: text for total count over gray bar
- textLayer: shows text for purple bar sub sum Bottom section: purple count bars
- barLayerBottom: merges copy of barLayer, shows purple bar per GWAS
- textLayer: shows text value of each bar
To set font size of axis labels and bars in multilayer plot:
- textLayer.mark.fontSize: 14
- textLayer.encoding.y.axis.labelFontSize: 14
- textLayerTotalTop.encoding.y.axis.fontSize: 14
The detailed spec for this page is available at: Notion ticket CMT-339.
The page consists of two tables, Table 1 and Table 2. Table 1 is essentially a groupby and rollup of Table 2. Table 2 data must be therefore be generated first. The data processing occurs in composable GenePageHelpers.js, which also contains the config info for the data tables (i.e., the headers plus watchers to handle the UI switches for showing color-coded variants, ensemble IDs, and the dataset names).
The Call Tree below indicates the overall flow from the top-level routines, getTable1Data and getTable2Data. It's a bit complicated, but I've tried to arrange it so that each function has a meaningful name, has a single responsibility, and returns a single object of interest. All the functions are local, except fetchData, which is a from the app-wide composable of the same name.
getTable2Data
getTableForGene
getRawData
fetchData
flattenData
gettableForTraitsVariants
getRawData
fetchData
flattenData
getTableGroupedSameTissue
getTableGroupedAnyTissue
getTable1Data
getTable1Objects
getGeneDetails
getRawData
fetchData
There is a single mechanism to force loading of new datasets. This is a watcher located on the gene page. It watches appStore.genePage.selectedGene. Whenever that value changes, the watcher triggers. If a valid gene is specified, then the values of r2 and h4 are reset to their defaults, and the loadData function is called. Assuming it completes succesfully, it updates the local variables table1Data and table2Data, which are linked to the associated data tables.
There is a second watcher that watches the store for changes to the r2 and h4 UI controls. Whenever either changes, data is reloaded.
There is a third watcher that watches the value of the 'gene' query parameter in the URL. Whenever it changes, the specified gene is pushed to the appStore.genePage.selectedGene, watcher #1 fires, and a new dataset is loaded.
This page allows loading of a gene of interest through several means:
- On page load, if gene present in URL, load data, update route with cleaned up gene (or remove as query param if bad gene specified)
- If select gene from dropdown: load data, update route
- If click gene on data table: load data, update route
- If route changes from back/fore buttons, load data, don't update route (browser does it)
A flag, updateRoute, was introduced into the appStore.genePage to track whether the route should be updated in the various scenarios. Also, 'load data' means to update appStore.genePage.selectedGene and let the watcher do its thing.
Late in development of this page, I experienced a situation where the gene list in column "Other Genes Same GWAS" in Table1, as well as the associated count column, weren't updating properly, sometimes. I refactored things such that:
- The
refvariables holding the table data on the gene page were switched to `shallowRef' types, which matches what I had done long ago in the data tables for the search, LZ, and Manhattan plot pages. - Originally, I had the composable return an object with entries for table1Data and table2Data. I refactored this into separate functions, which also helped clarify the code, and then call each in turn as appropriate in loadData on the gene page.
The existing TraitLabelGene component was enhanced to provide a link from the current page (search, LZ, Manhattan) to the gene page. (That component also displays a link to the CMD portal.) However, this component was not used in the display of genes on the gene page. Instead, a new simpler component was developed that just displays the gene symbol as a link to the gene page.
To add a tutorial to page,
First, create file with a set of step definitions (see examples in src/tutorials):
import steps from '@/tutorials/tutSearchPage'
Second, add the tutorial component to the page template, eg:
<h1>Search <TutorialOverlay ref="tutorial" :steps="steps" /></h1>
Third, import the step file and add a ref var for the tutorial:
import steps from '@/tutorials/tutSearchPageSteps'
const tutorial = ref()
Styling depends on a common style, D3FontDefaults, which is built dynmically in constants.js, based on config values in LZ2_DISPLAY_OPTIONS. That style is loaded by main.js and assigned to a root style element, thus is available globally without further declaration.
Originally I wrote a version of the plot export function for individual plots. Later, when writing the exporter for plot groups, I stumbled on a better library (html2canvas) that simplified the code and produced more accurate results. Later, I consolidated the code to use just the latter method for both individual plots and plot groups. For reference, this was done in commmit bab2c1711e469a425bb8540f18b5fd0caa0cf433, in case there is ever a need to resurrect the old method.
plotsContainer (div on host page)
mountEl (div in region plot manager, required for plot export))
plotContainer (div/ref in LZRegionPlot)
rootSVG (Vue variable in LZRegionPlot)
svg
rect (border)
g (headerGroup)
rect (fill color)
text (title)
text (hamburger icon, click handler)
thePlot (Vue variable)
g
xAxis
yAxisSignal
yAxisRecomb
plotGroup (Vue variable)
g (clipPath)
signalData
recombLine
genSigLine
The Multizoom page uses the standard SideBarLayout, with the standard FilterPanel. The main area also includes a separate panel of tools specific to that page, a grid component, a tooltip, and action menu, a busy overlay, and a standard data table. All runtime operational data is stored in the appStore, under the multizoom page key. In what follows, LZ2 stands for LocusZoom 2, and MZ stands for multizoom.
Files for this page exist in several places
- src/components/LZ2Components: main components: action menu, gene panel, region plot, tooltip
- src/components/misc widgets: mzGrid (grid system for displaying plots), MZPlot (overall wrapper for all plot types (region, gene, mock)), MZToolbox (tool panel)
- src/composables: plot component renderers (overall container, axes, scales); helper functions (lzPageHelpers, mzEventHandlers, mzGridHelpers)
The containment hierarchy is as follows:
Multizoom page
Filter panel
Main area
Multizoom toolbox
BusyOverlay (displayed during plot export)
LZ2Tooltip (displayed when hovering over a data point on a plot)
ActionMenu (displyed in response to click events on the plot hamburger menu and plot icons in the data table)
MZGrid
Corner (upper left)
Column headers
Row headers
Cells
Mock plot
Gene panel
Region plot
Data table
This is a panel that contains a set of controls for operating the app. Generally speaking, these controls initiate actions (e.g., delete all plots) or update values in the appStore. Those values in turn are watched by the plots, which redraw themselves on changes. For example, when the value of the plot region slider changes, the plots issue calls for data within that region and then redraw themselves.
The grid system is defined by MZGrid.vue. It defines the overall container using CSS grid features, and it uses values from the gridSettings in the appStore for this page for setting numbers of rows and columns, height and width of cells, etc. MZGrid draws the upper left corner, then the column headers, then the row headers, and then the cells. Each cell is an instance of MZPlot. The row and column headers have event handlers that respond to clicks; these events bubble up to the page file where they are dispatched for execution.
MZPlot's job is to figure out which type of plot to display in a given cell. At this point, there are three options:
- Mock plot: empty cell that displays cell coordinates and responds to click events
- Gene panel: plot displaying genes in the selected region
- Region plot: plot displaying colocalization data for the selected region
The mock plot is displayed by MZPlot directly; the other two by specific Vue components, LZ2RegionPlot and LZ2GenePanel.
The region plot is displayed by the LZ2RegionPlot component. There are two watchers that control operation. The first watcher renders a plot whenever the underlying data changes, or a UI control values change (e.g., the switches for showing the recomb line and genetic significance line). The second watcher triggers when changes occur in the selected LD reference variant, the conditional/marginal switch, or the zoom region. Then data is loaded from the back end and stored locally in the component. This then triggers the first watcher to redraw the plot. Each plot has its own watchers, so acts independently. If there are a lot of plots, then there are a lot of watchers.
Plot rendering is accomplished by a debounced function, debouncedRenderPlot, to avoid multiple triggers in succession. It in turns calls the actual render function, renderPlot. It in turn sets up data structures and other elements used by D3. renderPlot creates a root SVG element in which the plot is contained. It then calls a series of functions to build up the plot piece by piece. These functions are stored in a composable, LZ2Renderers, and include the following:
- renderBorder
- renderHeader
- renderXaxis
- renderYaxisSignal (-log 10 p-value)
- renderYaxisRecomb (recombination rate)
- renderPlotClipPath (establishes the area within which the plot is rendered, with no leakage outside this area)
- renderSignalData (the actual data points)
- renderRecombLine
- renderGenSignLine
- renderPlotIDBadge (used in an early iteration before we displayed cell coordinates, left in for use debugging)
One argument to renderSignalData is tooltipCallbacks. This is an object containing functions from the tooltip store; those functions display, update, and hide the tooltip seen when hovering over individual plot data points.
Within the rendering functions is a lot of D3 code for drawing points, lines, boxes, text, etc. One point of interest is the plot header, which displays a plot title and a 'hamburger' icon (three horizontal lines). An attribute is applied thus: .attr('data-action', 'hamburger-menu') When the icon is clicked, this has the effect of generating a click event called hamburger-menu. This is a JavaScript native click, not a Vue event, so must be handled separately. The event bubbles up to the MZGrid component, which is instantiated in the Multizoom page view, which therefore traps the event and dispatches it to an event handler, onNativeClick. I'll discuss MZ page event handlers in more detail below.
The gene panel should be viewed as an early draft product, not fully finished. Given greater time resources, it would have been more polished.
The gene panel is built in an analogous fashion as the region plot. Unlike the region plot, which has fixed horizontal and vertical dimensions, the gene panel has fixed horizontal but dynamically determined vertical dimensions. There is a computed property, dimensions, that does so. The gene panel has one watcher, which watches some of the same values in the appStore as the regionPlot (e.g., selectedLDRef, zoomRegion), as well as the gene data itself. Whereas the data for the region plot is stored within the component, the gene data for the gene panel is stored in the appStore. The reason is that a planned future enhancement would allow the list of genes to be supplemented by, for example, all genes shown in the data table, and all genes across all rendered plots. So the store is used as point of centralization.
The onMounted lifecycle hook has two actions: loadGeneData, and renderGenePlot. The render function is similar to the region plot's render function. A root SVG element is created, config data is computed, then render functions are called:
- renderBorder (same as used for region plot)
- renderGeneHeader
- renderXaxisGenePanel
- renderGenes
All these functions exist in the same LZ2Renderers file as those for the region plot. It would make sense to move the gene-specific ones into a separate composable.
The renderGenes function is at the heart of the gene panel. The code was derived from a Python Jupiter notebook proof-of-concept. First the list of genes is sorted by size, thus prioritizing labels on larger genes. Then the list is iterated, drawing in turn a span line, exons if any, the gene name, and so forth. Given more time, these would have been broken out into separate functions to aid comprehension, independence, and maintainability.
The action menu component displays when any of the following items is clicked:
- hamburger icon in a region plot or gene panel title
- row header
- column header
- plot icons in column 2 of the data table
The content of the action menu is determined dynamically, depending on the type. See files ActionMenuConfig.js and ActionMenu.vue in src/components/misc widgets/. The config file is a plain old JavaScript object, with a key for each type (column-header, datatable, gene-hamburger, etc). Within each key is a list of menu items. Each item has an id (not really used for anything), a type (currently: action, checkbox, or input), a label, an icon, and an event name. This file also has a function, getMenuItems, that returns the menu items for a given menu type. Another function, getActionMenuEventNames, returns a list of event names.
The Vue component (ActionMenu.vue) uses those functions to dynamically build and display a menu, given a user click on a particular component type. The list of event names is used to populate the defined Vue emit names (const emit = defineEmits(getActionMenuEventNames())). The corresponding name is then emitted as a Vue event when a button is clicked, the menu is closed, etc. The other function is used to dynamically build out the template with the associated menu item titles, icons, dividing lines, and input fields.
Upon user interaction, the ActionMenu generates the corresponding event, which bubbles up through the component hierarchy to the Multizoom page.
In the multizoom page, visibility of the ActionMenu is controlled by menuState.visible. menuState is an object in a composable, mzEventHandlers, which is where all the event handlers for this page live. The ActionMenu component currently has 22 event handlers to handle all the events for all the different menu elements. The names of the handlers are self-documenting (addGenePanel, exportPlot, etc.). By design, most of these are quite simple, offloading any complex functionality to helper functions. For example, here is the event handler for exporting a plot.
const onExportPlot = async () => {
menuState.value.visible = false
const plotDOMid = `plot_${storeMZpage.activePlotID}`
await mzGridHelpers.exportPlotContainer(plotDOMid, `Colocus_${plotDOMid}`)
}
First it hides the ActionMenu, then assembles the DOM ID of the plot to be exported, then calls the export function in the helpers composable.
This layering helps keep the code somewhat manageable; without it, the Multizoom page would be close to 1000 lines long.
The Multizoom page also includes the MZGrid component, which has its own set of events for rowClick, columnClick, and so forth. These follow the same pattern, with the handlers in mzEventHandlers, and the helpers in mzGridHelpers.
There are handlers with 'menu' in the name, e.g., columnMenu. These handlers respond to right-click events; this type of event is called 'contextmenu' in JavaScript. No functionality is attached to the menu events, but they are surfaced for potential use in the future.
The ActionMenu can be closed by clicking an action button on the menu, by pressing the Escape key, or by clicking anywhere outside the menu. This clicking outside is handled by a global Vue directive, clickOutside. This is defined in src/directives/clickOutside.js, and loaded in main.js for use anywhere in the app as 'click-outside'. It is used by including it in the template where it should apply, e.g., for the ActionMenu:
<div class="plot-action-menu" :style="menuStyle" v-click-outside="onCloseMenu">
So whenever a click-outside event bubbles through the app, it is trapped by the ActionMenu, the handler onCloseMenu is called (which sets menuState.visible to false), and the menu is closed.
If user clicks a hamburger icon in a plot, that generates a native click event, which bubbles up through the grid to the page, where it is handled by onNativeClick in the event handlers composable. It in turn figures out the type of event. We currently have three:
- hamburger-menu (from region plot)
- gene-hamburger-menu (from gene panel)
- gene-panel-background (from gene panel) (currently not used, but surfaced for potential use)
onNativeClick extracts the plotID and the event type from the underlying SVG's dataset (those 'data-*' attributes), then calls showPlotActionMenu with the plotID, menu type, and event type. showPlotActionMenu in turn figures out display location of the menu, then sets menuState to visible, and sets the display parameters and the context, which includes the plotID.
Then the ActionMenu template kicks into gear. ActionMenu has a computed property, visibleMenuItems, which gets the menu items to display from ActionMenuConfig.js, based on the menuType. Then ActionMenu's template iterates those items, displaying each according to its specified type (action, checkbox, input). The component then waits for events on its controls, or a 'click-away' event, in which case it goes hidden once again.
- First the user clicks a plot icon (a PlotNum component instance) in the data table.
- The PlotNum SFC emits on-plot-icon-click with args(plotID, slot, event). (The slot represents the signal and position, 1 or 2, of the icon in the row.)
- The DataTable traps that event, adds in the colocID and signal, and emits its own on-plot-icon-click with args(plotID, slot, event, colocID, signal).
- The MZPage view traps that event, adds menuType: 'datatable' to the args, and calls showPlotActionMenu.
- showPlotActionMenu sets visual params, then sets menuState, inserting colocID, signal, slot, and plotID into its context.
- When menu button (eg add) in ActionMenu clicked, ActionMenu emits an event corresponding to the type specified in ActionMenuConfig, eg 'add-plot'
- That event is trapped in MZPage, where the handler onAddPlot responds. First it extracts args.inputValue, which is what the user typed, as the value of cell, converts it, then calls mzGridHelpers.renderPlot with cell, colocID, signal, and slot.
Following are suggestions for improving functionlity and efficiency of this web application.
- Allow user editing of plot titles, with reset action to restore computed value
- Drag-and-drop plots to move on grid
- Add pan control slider on MZ page toolbox, with continuous values, going from -position(lead variant) to +position(lead variant), where default is centered
- Implement a locus compare plot and display in Multizoom page
- Add legend to display values of r^2 color codes
- Add a data manager to track data that has been loaded and reuse when possible instead of issuing reduncant calls to back end APIs
- Eliminate the Locuszoom page and supporting infrastructure
- Rewrite Vega plots on the QC pages using native D3
- Rewrite plot on Manhattan plot page using native D3
- Update help page to automatically generate sidebar links
- Simplify computation of appStore.dataTable.dirEffect (compute directly without use of isDirEffectReady flag)
- Redo the concordance hover on the data table to use a single instance (like the ActionMenu), instead of having an instance for every row in the table
- Update key libraries (Vue, Vite, Vuetify, Pinia, Vue Router): THIS SHOULD BE DONE REGULARLY TO KEEP UP WITH BUG AND SECURITY FIXES
- Update secondary libraries (most everything else, except the legacy D3 libs used by the LocusZoom page. The version of D3 used by the new functionality in this app is specified by the d3v7 key in package.json).