--- title: "Measuring AB Tests: SQL for p-value graphs in Redshift" description: SQL to do p-values and significance calculation in Redshift image: https://engineering.ezcater.com/hubfs/devblog/Screen%20Shot%202017-04-18%20at%209.44.39%20AM.png --- [![ezCater](https://engineering.ezcater.com/hs-fs/hubfs/RGB-ezCater-large.png?width=102&height=26&name=RGB-ezCater-large.png "ezCater")](http://www.ezcater.com) # Measuring AB Tests: SQL for p-value graphs in Redshift Jun 1 [Jeff Dwyer](https://engineering.ezcater.com/author/jeff-dwyer) So, in [deciding when an AB test is actionable using p-value graphs](https://engineering.ezcater.com/deciding-when-to-call-an-ab-test-p-value-graphs) we saw some cool charts that looked like: ![P-Value graph](https://engineering.ezcater.com/hs-fs/hubfs/devblog/Screen%20Shot%202017-04-18%20at%209.45.35%20AM.png?width=640&name=Screen%20Shot%202017-04-18%20at%209.45.35%20AM.png "P-Value graph")In these charts the green line is "cumulative p-value" and the red and blue track "conversions".  This example is a "definitely not significant" experiment. So how do we get our [redshift event schema](https://engineering.ezcater.com/ezexperiments-ab-testing-at-ezcater-part-1-event-taxonomy-warehousing) to lead to something that looks like this? Let's take a look at the SQL. First we need to find our Exposure and Conversion events.   Then we calculate the daily conversions.   Then we make them cumulative using some window functions.   Then some grouping up to make it tidy.    Finally we run a significance function for each day.   Here's the definition of the significance function from the wonderful [Periscope Blog](https://www.periscopedata.com/blog/ab-testing-in-redshift.html).   And that's it! We use the heck out of [Periscope](https://www.periscopedata.com) for visualizing and sharing these reports with others. We just need to configure things a little bit in there so that the conversion lines end up on a second y-axis.  ![Screenshot of Periscope](https://engineering.ezcater.com/hs-fs/hubfs/devblog/Screen%20Shot%202017-06-01%20at%201.19.09%20PM.png?width=640&name=Screen%20Shot%202017-06-01%20at%201.19.09%20PM.png "Screenshot of Periscope")   Tags: [experiments](https://engineering.ezcater.com/topic/experiments), [data warehousing](https://engineering.ezcater.com/topic/data-warehousing), [redshift](https://engineering.ezcater.com/topic/redshift), [sql](https://engineering.ezcater.com/topic/sql) ## Work With Us We’re always looking for highly skilled full stack engineers to help execute our technology goals while riding this rocket ship of growth. Our people are terrifically talented, friendly people who love what they do. And everyone is generous and kind, too — we have a strict no jerk policy. [![View ezCater Opportunities](https://no-cache.hubspot.com/cta/default/1703639/47d2a3d8-ffb7-4b28-8c3a-35bdb6d0605b.png)](https://cta-redirect.hubspot.com/cta/redirect/1703639/47d2a3d8-ffb7-4b28-8c3a-35bdb6d0605b) ### About ezCater We're the #1 online – and the only nationwide – marketplace for business catering in the United States. We make it easy to order food online for your office. From routine office lunches to offsite client meetings, from 5 to 2,000 people, we have a solution for you. ezCater connects business people with over 50,000 reliable local caterers and restaurants across the U.S. ### ezCater is hiring! We’re always looking for highly skilled full stack and iOS engineers to help execute our technology goals while riding this rocket ship of growth. Our people are terrifically talented straight-shooters who love what they do. And everyone is generous and kind, too — we have a strict no jerk policy. [![View ezCater Opportunities](https://no-cache.hubspot.com/cta/default/1703639/47d2a3d8-ffb7-4b28-8c3a-35bdb6d0605b.png)](https://cta-redirect.hubspot.com/cta/redirect/1703639/47d2a3d8-ffb7-4b28-8c3a-35bdb6d0605b) ### Recent Posts ### Posts by Topic - [experiments (6)](https://engineering.ezcater.com/topic/experiments) - [data warehousing (4)](https://engineering.ezcater.com/topic/data-warehousing) - [engineering (4)](https://engineering.ezcater.com/topic/engineering) - [ezcater (4)](https://engineering.ezcater.com/topic/ezcater) - [ruby (4)](https://engineering.ezcater.com/topic/ruby) - [activerecord (2)](https://engineering.ezcater.com/topic/activerecord) - [coding (2)](https://engineering.ezcater.com/topic/coding) - [development (2)](https://engineering.ezcater.com/topic/development) - [github (2)](https://engineering.ezcater.com/topic/github) - [javascript (2)](https://engineering.ezcater.com/topic/javascript) - [ltv (2)](https://engineering.ezcater.com/topic/ltv) - [machine learning (2)](https://engineering.ezcater.com/topic/machine-learning) - [rails (2)](https://engineering.ezcater.com/topic/rails) - [react (2)](https://engineering.ezcater.com/topic/react) - [redshift (2)](https://engineering.ezcater.com/topic/redshift) - [women\_in\_tech (2)](https://engineering.ezcater.com/topic/women_in_tech) - [active record (1)](https://engineering.ezcater.com/topic/active-record) - [creating (1)](https://engineering.ezcater.com/topic/creating) - [culture (1)](https://engineering.ezcater.com/topic/culture) - [database (1)](https://engineering.ezcater.com/topic/database) - [design (1)](https://engineering.ezcater.com/topic/design) - [devops (1)](https://engineering.ezcater.com/topic/devops) - [dynamodb (1)](https://engineering.ezcater.com/topic/dynamodb) - [growth (1)](https://engineering.ezcater.com/topic/growth) - [hubspot (1)](https://engineering.ezcater.com/topic/hubspot) - [jest (1)](https://engineering.ezcater.com/topic/jest) - [kafka (1)](https://engineering.ezcater.com/topic/kafka) - [kubernetes (1)](https://engineering.ezcater.com/topic/kubernetes) - [migrations (1)](https://engineering.ezcater.com/topic/migrations) - [snapshot testing (1)](https://engineering.ezcater.com/topic/snapshot-testing) - [snowflake (1)](https://engineering.ezcater.com/topic/snowflake) - [sql (1)](https://engineering.ezcater.com/topic/sql) - [time (1)](https://engineering.ezcater.com/topic/time) - [time zones (1)](https://engineering.ezcater.com/topic/time-zones) - [timeofday (1)](https://engineering.ezcater.com/topic/timeofday) - [unit testing (1)](https://engineering.ezcater.com/topic/unit-testing) - [values (1)](https://engineering.ezcater.com/topic/values) see all ### Subscribe to Email Updates © 2026 ezCater, Inc. | [Terms & Conditions](https://www.ezcater.com/terms_of_use) | [Privacy](https://www.ezcater.com/privacy_policy)