Thank you for the nice article and good appropriate examples, especially that of frequency distribution. its population shown for reference. As it happens with ROC Curve and ROC AUC, we cannot calculate the KS for a multiclass problem without transforming that into a binary classification problem. I already referred the posts here and here but they are different and doesn't answer my problem. You can use the KS2 test to compare two samples. Already have an account? Hypotheses for a two independent sample test. To test the goodness of these fits, I test the with scipy's ks-2samp test. that the two samples came from the same distribution. If the sample sizes are very nearly equal it's pretty robust to even quite unequal variances. On the image above the blue line represents the CDF for Sample 1 (F1(x)), and the green line is the CDF for Sample 2 (F2(x)). La prueba de Kolmogorov-Smirnov, conocida como prueba KS, es una prueba de hiptesis no paramtrica en estadstica, que se utiliza para detectar si una sola muestra obedece a una determinada distribucin o si dos muestras obedecen a la misma distribucin. Then we can calculate the p-value with KS distribution for n = len(sample) by using the Survival Function of the KS distribution scipy.stats.kstwo.sf[3]: The samples norm_a and norm_b come from a normal distribution and are really similar. Charles. There is clearly visible that the fit with two gaussians is better (as it should be), but this doesn't reflect in the KS-test. Your question is really about when to use the independent samples t-test and when to use the Kolmogorov-Smirnov two sample test; the fact of their implementation in scipy is entirely beside the point in relation to that issue (I'd remove that bit). Nevertheless, it can be a little hard on data some times. It seems straightforward, give it: (A) the data; (2) the distribution; and (3) the fit parameters. empirical CDFs (ECDFs) of the samples. Interpreting ROC Curve and ROC AUC for Classification Evaluation. https://ocw.mit.edu/courses/18-443-statistics-for-applications-fall-2006/pages/lecture-notes/, Wessel, P. (2014)Critical values for the two-sample Kolmogorov-Smirnov test(2-sided), University Hawaii at Manoa (SOEST) Master in Deep Learning for CV | Data Scientist @ Banco Santander | Generative AI Researcher | http://viniciustrevisan.com/, print("Positive class with 50% of the data:"), print("Positive class with 10% of the data:"). Are your distributions fixed, or do you estimate their parameters from the sample data? As I said before, the same result could be obtained by using the scipy.stats.ks_1samp() function: The two-sample KS test allows us to compare any two given samples and check whether they came from the same distribution. Time arrow with "current position" evolving with overlay number. rev2023.3.3.43278. Stack Exchange network consists of 181 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. dosage acide sulfurique + soude; ptition assemble nationale edf ks_2samp interpretation. One such test which is popularly used is the Kolmogorov Smirnov Two Sample Test (herein also referred to as "KS-2"). scipy.stats.ks_2samp. is the maximum (most positive) difference between the empirical The sample norm_c also comes from a normal distribution, but with a higher mean. In most binary classification problems we use the ROC Curve and ROC AUC score as measurements of how well the model separates the predictions of the two different classes. To learn more, see our tips on writing great answers. It is distribution-free. The difference between the phonemes /p/ and /b/ in Japanese, Acidity of alcohols and basicity of amines. iter = # of iterations used in calculating an infinite sum (default = 10) in KDIST and KINV, and iter0 (default = 40) = # of iterations used to calculate KINV. Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. To build the ks_norm(sample)function that evaluates the KS 1-sample test for normality, we first need to calculate the KS statistic comparing the CDF of the sample with the CDF of the normal distribution (with mean = 0 and variance = 1). Is it possible to create a concave light? By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. And how does data unbalance affect KS score? Ejemplo 1: Prueba de Kolmogorov-Smirnov de una muestra Perhaps this is an unavoidable shortcoming of the KS test. There is a benefit for this approach: the ROC AUC score goes from 0.5 to 1.0, while KS statistics range from 0.0 to 1.0. underlying distributions, not the observed values of the data. A Medium publication sharing concepts, ideas and codes. The only problem is my results don't make any sense? Browse other questions tagged, Start here for a quick overview of the site, Detailed answers to any questions you might have, Discuss the workings and policies of this site. [I'm using R.]. You reject the null hypothesis that the two samples were drawn from the same distribution if the p-value is less than your significance level. From the docs scipy.stats.ks_2samp This is a two-sided test for the null hypothesis that 2 independent samples are drawn from the same continuous distribution scipy.stats.ttest_ind This is a two-sided test for the null hypothesis that 2 independent samples have identical average (expected) values. I should also note that the KS test tell us whether the two groups are statistically different with respect to their cumulative distribution functions (CDF), but this may be inappropriate for your given problem. Hello Ramnath, What can a lawyer do if the client wants him to be acquitted of everything despite serious evidence? finds that the median of x2 to be larger than the median of x1, I figured out answer to my previous query from the comments. Is there a proper earth ground point in this switch box? less: The null hypothesis is that F(x) >= G(x) for all x; the When txt = FALSE (default), if the p-value is less than .01 (tails = 2) or .005 (tails = 1) then the p-value is given as 0 and if the p-value is greater than .2 (tails = 2) or .1 (tails = 1) then the p-value is given as 1. A place where magic is studied and practiced? Therefore, for each galaxy cluster, I have two distributions that I want to compare. Histogram overlap? The 2 sample KolmogorovSmirnov test of distribution for two different samples. On the equivalence between Kolmogorov-Smirnov and ROC curve metrics for binary classification. The single-sample (normality) test can be performed by using the scipy.stats.ks_1samp function and the two-sample test can be done by using the scipy.stats.ks_2samp function. I followed all steps from your description and I failed on a stage of D-crit calculation. The test only really lets you speak of your confidence that the distributions are different, not the same, since the test is designed to find alpha, the probability of Type I error. When I compare their histograms, they look like they are coming from the same distribution. [2] Scipy Api Reference. What is the point of Thrower's Bandolier? you cannot reject the null hypothesis that the distributions are the same). Is it suspicious or odd to stand by the gate of a GA airport watching the planes? Figure 1 Two-sample Kolmogorov-Smirnov test. Theoretically Correct vs Practical Notation, Topological invariance of rational Pontrjagin classes for non-compact spaces. If R2 is omitted (the default) then R1 is treated as a frequency table (e.g. The distribution naturally only has values >= 0. of two independent samples. This is a very small value, close to zero. "We, who've been connected by blood to Prussia's throne and people since Dppel". The a and b parameters are my sequence of data or I should calculate the CDFs to use ks_2samp? correction de texte je n'aimerais pas tre un mari. I think. It is most suited to Browse other questions tagged, Start here for a quick overview of the site, Detailed answers to any questions you might have, Discuss the workings and policies of this site. . Imagine you have two sets of readings from a sensor, and you want to know if they come from the same kind of machine. My code is GPL licensed, can I issue a license to have my code be distributed in a specific MIT licensed project? The overlap is so intense on the bad dataset that the classes are almost inseparable. The KS test (as will all statistical tests) will find differences from the null hypothesis no matter how small as being "statistically significant" given a sufficiently large amount of data (recall that most of statistics was developed during a time when data was scare, so a lot of tests seem silly when you are dealing with massive amounts of data). All of them measure how likely a sample is to have come from a normal distribution, with a related p-value to support this measurement. By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. If method='asymp', the asymptotic Kolmogorov-Smirnov distribution is used to compute an approximate p-value. greater: The null hypothesis is that F(x) <= G(x) for all x; the Partner is not responding when their writing is needed in European project application, Short story taking place on a toroidal planet or moon involving flying, Topological invariance of rational Pontrjagin classes for non-compact spaces. A priori, I expect that the KS test returns me the following result: "ehi, the two distributions come from the same parent sample". the empirical distribution function of data2 at Is it correct to use "the" before "materials used in making buildings are"? If the KS statistic is large, then the p-value will be small, and this may Share Cite Follow answered Mar 12, 2020 at 19:34 Eric Towers 65.5k 3 48 115 In order to quantify the difference between the two distributions with a single number, we can use Kolmogorov-Smirnov distance. For each photometric catalogue, I performed a SED fitting considering two different laws. In this case, The approach is to create a frequency table (range M3:O11 of Figure 4) similar to that found in range A3:C14 of Figure 1, and then use the same approach as was used in Example 1. Charles. Go to https://real-statistics.com/free-download/ However, the test statistic or p-values can still be interpreted as a distance measure. scipy.stats.ks_2samp(data1, data2) [source] Computes the Kolmogorov-Smirnov statistic on 2 samples. It only takes a minute to sign up. In a simple way we can define the KS statistic for the 2-sample test as the greatest distance between the CDFs (Cumulative Distribution Function) of each sample. Can I tell police to wait and call a lawyer when served with a search warrant? The quick answer is: you can use the 2 sample Kolmogorov-Smirnov (KS) test, and this article will walk you through this process. The calculations dont assume that m and n are equal. This isdone by using the Real Statistics array formula =SortUnique(J4:K11) in range M4:M10 and then inserting the formula =COUNTIF(J$4:J$11,$M4) in cell N4 and highlighting the range N4:O10 followed by, Linear Algebra and Advanced Matrix Topics, Descriptive Stats and Reformatting Functions, https://ocw.mit.edu/courses/18-443-statistics-for-applications-fall-2006/pages/lecture-notes/, https://www.webdepot.umontreal.ca/Usagers/angers/MonDepotPublic/STT3500H10/Critical_KS.pdf, https://real-statistics.com/free-download/, https://www.real-statistics.com/binomial-and-related-distributions/poisson-distribution/, Wilcoxon Rank Sum Test for Independent Samples, Mann-Whitney Test for Independent Samples, Data Analysis Tools for Non-parametric Tests. how to select best fit continuous distribution from two Goodness-to-fit tests? It's testing whether the samples come from the same distribution (Be careful it doesn't have to be normal distribution). This is a two-sided test for the null hypothesis that 2 independent samples are drawn from the same continuous distribution. There is even an Excel implementation called KS2TEST. two-sided: The null hypothesis is that the two distributions are identical, F (x)=G (x) for all x; the alternative is that they are not identical. To do that, I have two functions, one being a gaussian, and one the sum of two gaussians. If p<0.05 we reject the null hypothesis and assume that the sample does not come from a normal distribution, as it happens with f_a. How to show that an expression of a finite type must be one of the finitely many possible values? For example, Is it possible to create a concave light? with n as the number of observations on Sample 1 and m as the number of observations in Sample 2. If you assume that the probabilities that you calculated are samples, then you can use the KS2 test. This test is really useful for evaluating regression and classification models, as will be explained ahead. @meri: there's an example on the page I linked to. 31 Mays 2022 in paradise hills what happened to amarna Yorum yaplmam 0 . Further, just because two quantities are "statistically" different, it does not mean that they are "meaningfully" different. In the figure I showed I've got 1043 entries, roughly between $-300$ and $300$. identical, F(x)=G(x) for all x; the alternative is that they are not Jr., The Significance Probability of the Smirnov We can also check the CDFs for each case: As expected, the bad classifier has a narrow distance between the CDFs for classes 0 and 1, since they are almost identical. null and alternative hypotheses. This is a two-sided test for the null hypothesis that 2 independent samples are drawn from the same continuous distribution. It seems like you have listed data for two samples, in which case, you could use the two K-S test, but Is this the most general expression of the KS test ? While the algorithm itself is exact, numerical Help please! In the same time, we observe with some surprise . This test compares the underlying continuous distributions F(x) and G(x) I am currently working on a binary classification problem with random forests, neural networks etc. Both ROC and KS are robust to data unbalance. Follow Up: struct sockaddr storage initialization by network format-string. It seems to assume that the bins will be equally spaced. I only understood why I needed to use KS when I started working in a place that used it. Note that the alternative hypotheses describe the CDFs of the is about 1e-16. How do you get out of a corner when plotting yourself into a corner. KS Test is also rather useful to evaluate classification models, and I will write a future article showing how can we do that. I have Two samples that I want to test (using python) if they are drawn from the same distribution. By clicking Accept all cookies, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. The p-values are wrong if the parameters are estimated. The procedure is very similar to the, The approach is to create a frequency table (range M3:O11 of Figure 4) similar to that found in range A3:C14 of Figure 1, and then use the same approach as was used in Example 1. Finite abelian groups with fewer automorphisms than a subgroup. Is there a proper earth ground point in this switch box? The Kolmogorov-Smirnov test may also be used to test whether two underlying one-dimensional probability distributions differ. Is a PhD visitor considered as a visiting scholar? Suppose that the first sample has size m with an observed cumulative distribution function of F(x) and that the second sample has size n with an observed cumulative distribution function of G(x). ks_2samp Notes There are three options for the null and corresponding alternative hypothesis that can be selected using the alternative parameter. For example, perhaps you only care about whether the median outcome for the two groups are different. The best answers are voted up and rise to the top, Start here for a quick overview of the site, Detailed answers to any questions you might have, Discuss the workings and policies of this site. Defines the null and alternative hypotheses. I explain this mechanism in another article, but the intuition is easy: if the model gives lower probability scores for the negative class, and higher scores for the positive class, we can say that this is a good model. Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Your samples are quite large, easily enough to tell the two distributions are not identical, in spite of them looking quite similar. So let's look at largish datasets Had a read over it and it seems indeed a better fit. 1. why is kristen so fat on last man standing . Check out the Wikipedia page for the k-s test. How to prove that the supernatural or paranormal doesn't exist? How to follow the signal when reading the schematic? Use MathJax to format equations. scipy.stats. The Kolmogorov-Smirnov statistic D is given by. rev2023.3.3.43278. (this might be a programming question). from scipy.stats import ks_2samp s1 = np.random.normal(loc = loc1, scale = 1.0, size = size) s2 = np.random.normal(loc = loc2, scale = 1.0, size = size) (ks_stat, p_value) = ks_2samp(data1 = s1, data2 = s2) . E-Commerce Site for Mobius GPO Members ks_2samp interpretation. How do I make function decorators and chain them together? If you preorder a special airline meal (e.g. measured at this observation. Your home for data science. Charles. but the Wilcox test does find a difference between the two samples. distribution, sample sizes can be different. * specifically for its level to be correct, you need this assumption when the null hypothesis is true. When txt = TRUE, then the output takes the form < .01, < .005, > .2 or > .1. Hi Charles, thank you so much for these complete tutorials about Kolmogorov-Smirnov tests. More precisly said You reject the null hypothesis that the two samples were drawn from the same distribution if the p-value is less than your significance level. Why are non-Western countries siding with China in the UN? Also, why are you using the two-sample KS test? Even if ROC AUC is the most widespread metric for class separation, it is always useful to know both. Does Counterspell prevent from any further spells being cast on a given turn? [1] Scipy Api Reference. If your bins are derived from your raw data, and each bin has 0 or 1 members, this assumption will almost certainly be false. ks_2samp interpretation. If method='asymp', the asymptotic Kolmogorov-Smirnov distribution is scipy.stats.ks_2samp(data1, data2, alternative='two-sided', mode='auto') [source] . If the the assumptions are true, the t-test is good at picking up a difference in the population means. Assuming that one uses the default assumption of identical variances, the second test seems to be testing for identical distribution as well. Why do many companies reject expired SSL certificates as bugs in bug bounties? The statistic Why do small African island nations perform better than African continental nations, considering democracy and human development? Hello Ramnath, A p_value of pvalue=0.55408436218441004 is saying that the normal and gamma sampling are from the same distirbutions? On the x-axis we have the probability of an observation being classified as positive and on the y-axis the count of observations in each bin of the histogram: The good example (left) has a perfect separation, as expected. You should get the same values for the KS test when (a) your bins are the raw data or (b) your bins are aggregates of the raw data where each bin contains exactly the same values. Learn more about Stack Overflow the company, and our products. Am I interpreting the test incorrectly? By clicking Accept all cookies, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. We can now evaluate the KS and ROC AUC for each case: The good (or should I say perfect) classifier got a perfect score in both metrics. Ah. alternative is that F(x) < G(x) for at least one x. Taking m = 2 as the mean of Poisson distribution, I calculated the probability of @O.rka But, if you want my opinion, using this approach isn't entirely unreasonable. Also, I'm pretty sure the KT test is only valid if you have a fully specified distribution in mind beforehand. Using K-S test statistic, D max can I test the comparability of the above two sets of probabilities? Topological invariance of rational Pontrjagin classes for non-compact spaces. The procedure is very similar to the One Kolmogorov-Smirnov Test(see alsoKolmogorov-SmirnovTest for Normality). Statistics for applications In the latter case, there shouldn't be a difference at all, since the sum of two normally distributed random variables is again normally distributed. We cannot consider that the distributions of all the other pairs are equal. Now you have a new tool to compare distributions. The nature of simulating nature: A Q&A with IBM Quantum researcher Dr. Jamie We've added a "Necessary cookies only" option to the cookie consent popup. . Why do many companies reject expired SSL certificates as bugs in bug bounties? [2] Scipy Api Reference. OP, what do you mean your two distributions? Therefore, we would As for the Kolmogorov-Smirnov test for normality, we reject the null hypothesis (at significance level ) if Dm,n > Dm,n, where Dm,n,is the critical value. We can now perform the KS test for normality in them: We compare the p-value with the significance. For each galaxy cluster, I have a photometric catalogue. scipy.stats.ks_1samp. On the scipy docs If the KS statistic is small or the p-value is high, then we cannot reject the hypothesis that the distributions of the two samples are the same. The following options are available (default is auto): auto : use exact for small size arrays, asymp for large, exact : use exact distribution of test statistic, asymp : use asymptotic distribution of test statistic. The f_a sample comes from a F distribution. What is the point of Thrower's Bandolier? Is there a reason for that? Accordingly, I got the following 2 sets of probabilities: Poisson approach : 0.135 0.271 0.271 0.18 0.09 0.053 Use MathJax to format equations. Interpretting the p-value when inverting the null hypothesis. rev2023.3.3.43278. For example I have two data sets for which the p values are 0.95 and 0.04 for the ttest(tt_equal_var=True) and the ks test, respectively. Real Statistics Function: The following functions are provided in the Real Statistics Resource Pack: KSDIST(x, n1, n2, b, iter) = the p-value of the two-sample Kolmogorov-Smirnov test at x (i.e. Can airtags be tracked from an iMac desktop, with no iPhone? But who says that the p-value is high enough? Check it out! Is it correct to use "the" before "materials used in making buildings are"? If interp = TRUE (default) then harmonic interpolation is used; otherwise linear interpolation is used. Cell G14 contains the formula =MAX(G4:G13) for the test statistic and cell G15 contains the formula =KSINV(G1,B14,C14) for the critical value. Two-sample Kolmogorov-Smirnov Test in Python Scipy, scipy kstest not consistent over different ranges. How to interpret p-value of Kolmogorov-Smirnov test (python)? To learn more, see our tips on writing great answers. +1 if the empirical distribution function of data1 exceeds Can I still use K-S or not? What is a word for the arcane equivalent of a monastery? https://en.m.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test, soest.hawaii.edu/wessel/courses/gg313/Critical_KS.pdf, We've added a "Necessary cookies only" option to the cookie consent popup, Kolmogorov-Smirnov test statistic interpretation with large samples. draw two independent samples s1 and s2 of length 1000 each, from the same continuous distribution. I have some data which I want to analyze by fitting a function to it. Is it plausible for constructed languages to be used to affect thought and control or mold people towards desired outcomes? Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. For Example 1, the formula =KS2TEST(B4:C13,,TRUE) inserted in range F21:G25 generates the output shown in Figure 2. We can calculate the distance between the two datasets as the maximum distance between their features. Can you please clarify the following: in KS two sample example on Figure 1, Dcrit in G15 cell uses B/C14 cells, which are not n1/n2 (they are both = 10) but total numbers of men/women used in the data (80 and 62). To this histogram I make my two fits (and eventually plot them, but that would be too much code). It is widely used in BFSI domain. I think I know what to do from here now. So the null-hypothesis for the KT test is that the distributions are the same. As seen in the ECDF plots, x2 (brown) stochastically dominates In this case, probably a paired t-test is appropriate, or if the normality assumption is not met, the Wilcoxon signed-ranks test could be used. does elena end up with damon; mental health association west orange, nj. Example 1: One Sample Kolmogorov-Smirnov Test. Charles. The original, where the positive class has 100% of the original examples (500), A dataset where the positive class has 50% of the original examples (250), A dataset where the positive class has only 10% of the original examples (50). rev2023.3.3.43278. par | Juil 2, 2022 | mitchell wesley carlson charged | justin strauss net worth | Juil 2, 2022 | mitchell wesley carlson charged | justin strauss net worth There cannot be commas, excel just doesnt run this command. by. Next, taking Z = (X -m)/m, again the probabilities of P(X=0), P(X=1 ), P(X=2), P(X=3), P(X=4), P(X >=5) are calculated using appropriate continuity corrections. This is the same problem that you see with histograms. I am sure I dont output the same value twice, as the included code outputs the following: (hist_cm is the cumulative list of the histogram points, plotted in the upper frames). Performs the two-sample Kolmogorov-Smirnov test for goodness of fit. This means at a 5% level of significance, I can reject the null hypothesis that distributions are identical. How can I proceed. By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. My only concern is about CASE 1, where the p-value is 0.94, and I do not know if it is a problem or not. can discern that the two samples aren't from the same distribution. Suppose we wish to test the null hypothesis that two samples were drawn How do I align things in the following tabular environment? So, heres my follow-up question. Fitting distributions, goodness of fit, p-value. KS2TEST(R1, R2, lab, alpha, b, iter0, iter) is an array function that outputs a column vector with the values D-stat, p-value, D-crit, n1, n2 from the two-sample KS test for the samples in ranges R1 and R2, where alpha is the significance level (default = .05) and b, iter0, and iter are as in KSINV. It only takes a minute to sign up. I am not familiar with the Python implementation and so I am unable to say why there is a difference. Does a barbarian benefit from the fast movement ability while wearing medium armor? Sign up for free to join this conversation on GitHub . KS2TEST gives me a higher d-stat value than any of the differences between cum% A and cum%B, The max difference is 0.117 Is it possible to do this with Scipy (Python)? For 'asymp', I leave it to someone else to decide whether ks_2samp truly uses the asymptotic distribution for one-sided tests. Using Scipy's stats.kstest module for goodness-of-fit testing. Charles. scipy.stats.kstwo. from a couple of slightly different distributions and see if the K-S two-sample test alternative. were not drawn from the same distribution. scipy.stats.ks_1samp. against the null hypothesis. As Stijn pointed out, the k-s test returns a D statistic and a p-value corresponding to the D statistic. Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. The null hypothesis is H0: both samples come from a population with the same distribution. [5] Trevisan, V. Interpreting ROC Curve and ROC AUC for Classification Evaluation. By clicking Accept all cookies, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. If so, it seems that if h(x) = f(x) g(x), then you are trying to test that h(x) is the zero function. Am I interpreting this incorrectly? In fact, I know the meaning of the 2 values D and P-value but I can't see the relation between them. Connect and share knowledge within a single location that is structured and easy to search. The statistic is the maximum absolute difference between the Context: I performed this test on three different galaxy clusters. We choose a confidence level of 95%; that is, we will reject the null Default is two-sided. So I conclude they are different but they clearly aren't? that is, the probability under the null hypothesis of obtaining a test Learn more about Stack Overflow the company, and our products. The two-sample t-test assumes that the samples are drawn from Normal distributions with identical variances*, and is a test for whether the population means differ. exactly the same, some might say a two-sample Wilcoxon test is Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. For business teams, it is not intuitive to understand that 0.5 is a bad score for ROC AUC, while 0.75 is only a medium one. I want to know when sample sizes are not equal (in case of the country) then which formulae i can use manually to find out D statistic / Critical value. What is the correct way to screw wall and ceiling drywalls? 11 Jun 2022. [1] Adeodato, P. J. L., Melo, S. M. On the equivalence between Kolmogorov-Smirnov and ROC curve metrics for binary classification. Column E contains the cumulative distribution for Men (based on column B), column F contains the cumulative distribution for Women, and column G contains the absolute value of the differences.