Some preliminaries

If you have not already, recommend reviewing the following posts in order to reproduce the results shown below.

This post is created in RStudio using R notebook format. For an overview of R notebook markdowns, a good resource is: https://rmarkdown.rstudio.com/authoring_basics.html

Background

In this example we develop a basic regression model to explain the variation in the compressive strength of concrete. Concrete is the literal building block that has shaped modern living. Figuring out what makes it stronger is a very important topic in construction.

We hypothesize that the compressive strength of concrete is a function of the age and the mixture of ingredients.

The data that we use in this example can be downloaded here: https://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength

Modeling

The process of linear regression modeling broadly involves 1) exploring the data, 2) variable selection 3) estimating the parameters of the model, and 4) evaluating the model. And not necessarily in that order! I illustrate a few of these steps below.

Data Exploration

First, we read the data in. Notice the “fileEncoding” attribute. I had to include it because we were reading in a pesky hidden character at the start of the file.

dat = read.csv("D:/Data/PredictiveModeler/UCI_Data/ConcreteCompressive/Concrete_Data.csv", header = TRUE, fileEncoding="UTF-8-BOM")

We can use head(dat) command to display & inspect the top rows of the data. You can click on button on the top-right to show more columns. The variable that we wish to predict/estimate/explain the variation in, is the Concrete Compressive Strength, or “CCS” (i.e. the last column).

We check the number of rows and columns in the data, as well as how they are distributed using the commands below.

nrow(dat) #number of rows
[1] 1030
ncol(dat) #number of columns
[1] 9
summary(dat) #min / mean / median etc.
   C1.Cement     C2.Blast.Furnace.Slag   C3.Fly.Ash        C4.Water     C5.Superplasticizer C6.Coarse.Aggregate
 Min.   :102.0   Min.   :  0.0         Min.   :  0.00   Min.   :121.8   Min.   : 0.000      Min.   : 801.0     
 1st Qu.:192.4   1st Qu.:  0.0         1st Qu.:  0.00   1st Qu.:164.9   1st Qu.: 0.000      1st Qu.: 932.0     
 Median :272.9   Median : 22.0         Median :  0.00   Median :185.0   Median : 6.400      Median : 968.0     
 Mean   :281.2   Mean   : 73.9         Mean   : 54.19   Mean   :181.6   Mean   : 6.205      Mean   : 972.9     
 3rd Qu.:350.0   3rd Qu.:142.9         3rd Qu.:118.30   3rd Qu.:192.0   3rd Qu.:10.200      3rd Qu.:1029.4     
 Max.   :540.0   Max.   :359.4         Max.   :200.10   Max.   :247.0   Max.   :32.200      Max.   :1145.0     
 C7.Fine.Aggregate      Age              CCS       
 Min.   :594.0     Min.   :  1.00   Min.   : 2.33  
 1st Qu.:731.0     1st Qu.:  7.00   1st Qu.:23.71  
 Median :779.5     Median : 28.00   Median :34.45  
 Mean   :773.6     Mean   : 45.66   Mean   :35.82  
 3rd Qu.:824.0     3rd Qu.: 56.00   3rd Qu.:46.13  
 Max.   :992.6     Max.   :365.00   Max.   :82.60  

One of the first things to do is to see how our dependent variable (CCS) is distributed. We can create a quick histogram using the command: hist(dat$CCS). This isn’t a test of anything, but just helps us understand the range of our dependent values, and how those values are distributed along that range.

Next, we wish to see how our independent variables are correlated with our dependent variable, CCS. Strong positive or negative correlations with the dependent variable suggest that the independent variable might be important in terms of explaining the dependent variable’s variation. Strong correlations amongst the independent variables suggest that multicollinearity (MC) might be an issue in the modeling. Technically, multicollinearity implies that an independent variable might be predicted linearly by one or more other indepdent variables to a high degree of accuracy. MC is an issue because it makes the coefficient estimates highly dependent upon which variables are included in the model and makes it difficult to properly interpret them (while not increasing the predictive power of the model in proportion to the disadvantages it causes!)*.

We use a new library called “psych” in order to create these plots, as below. The code can be edited to see how the variables are distributed and how they are correlated with each other. If you do not have the library installed, you will need to do so by going to Tools > Install Packages… in your RStudio program.

*For more information: https://newonlinecourses.science.psu.edu/stat501/node/346/

library(psych)
pairs.panels(dat[c(9,1,7:8)], 
             method = "pearson", # correlation method
             hist.col = "#00AFBB",
             density = TRUE,  # show density plots
             ellipses = TRUE # show correlation ellipses
             )

Key Assumptions: Linear Regression

The key assumptions of linear regression that we want to test include:

  1. Linearity & Additivity: An assumption that the dependent variable is a linear and additive function of the independent variables.
  2. Independence of errors (residuals): The errors are not correlated.
  3. Constant variance of errors (also called by a needlessly complicated name: homoscedasticity): The variance of the residuals should be approximately equal across the range of the dependent variable (CCS in our case)
  4. Errors are distributed normally: Generally less important than the assumptions above, we ideally want to see the residuals distribtued normally.

Functional Form

We are skipping this step here, as we are assuming a linear relationship between the independent variables and the dependent variable. I will make a post later on how to estimate a functional form given data.

Parameter Estimation

Next, we fit a linear model using all of the independent variables that we have. We can do so using the following command:

mfull=lm(CCS~.,data=dat)

We call our model “mfull” (or full model). We regress the dependent variable CCS on all other variables using the “~.” command. We can also select certain variables using “CCS~Age+C1.Cement” etc.

Next, we display a summary of the fitted model.

summary(mfull)

Call:
lm(formula = CCS ~ ., data = dat)

Residuals:
    Min      1Q  Median      3Q     Max 
-28.654  -6.302   0.703   6.569  34.450 

Coefficients:
                        Estimate Std. Error t value Pr(>|t|)    
(Intercept)           -23.331214  26.585504  -0.878 0.380372    
C1.Cement               0.119804   0.008489  14.113  < 2e-16 ***
C2.Blast.Furnace.Slag   0.103866   0.010136  10.247  < 2e-16 ***
C3.Fly.Ash              0.087934   0.012583   6.988 5.02e-12 ***
C4.Water               -0.149918   0.040177  -3.731 0.000201 ***
C5.Superplasticizer     0.292225   0.093424   3.128 0.001810 ** 
C6.Coarse.Aggregate     0.018086   0.009392   1.926 0.054425 .  
C7.Fine.Aggregate       0.020190   0.010702   1.887 0.059491 .  
Age                     0.114222   0.005427  21.046  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 10.4 on 1021 degrees of freedom
Multiple R-squared:  0.6155,    Adjusted R-squared:  0.6125 
F-statistic: 204.3 on 8 and 1021 DF,  p-value: < 2.2e-16

Here are a few key observations from the linear regression summary:

R-Square

We have an R-square of 61.55%, and an adjusted R-Square of about the same. This is good, because the adjusted R-Square adjusts the metric for the number of predictors in the model. The adjusted R-Square is always lower than R-Square, and a big difference hints that the model might have too many predictors than is required, and the following section on variable selection becomes more important!

More information:

Overall F-statistic / p-value

The F-test in a multivariable linear regression helps us determine whether any of the independent variables is significant. Note that it does not tell us that all or most of the variables are significant, for that we have to inspect other diagnostics (see below).

The overall F-stat produces a very tiny p-value, so we can reject the null hypotheses that all of the coefficients are in fact 0 (i.e. no variable is significant). Generally speaking, we want this value to be below 0.05.

More information:

Residual Standard Error

The model has a Residual Standard Error (RSE) of 10.4. This, along with the R-square, is a measure of the goodness of fit of the model.The Standard Error (SE) tells us the average error between the data points and the fitted line. It is calculated as:

\[ RSE = \hat{\sigma} = \sqrt{\frac{\Sigma_{i=1}^{n}{(Y_i -\hat{Y_i})^2}} {n-k-1} } \] Where \(k\) is the number of predictors in the model and \(n\) is the number of observations. In the case of our model, an RSE of 10.4 is telling us that on average the difference between the reported data and fitted line is 10.4. If the residuals are distributed normally, then about two-thirds of the errors are in the \(\pm10.4\) range and about 95% of the errors are in the \(\pm20.8\) range.

Variable significance tests

In the summary output above, we have for each variable: 1) the coefficient estimate, 2) the standard error, 3) the t-value, and 4) the p-value. If the p-value is small enough (e.g. less than 0.05), then we tend to reject the null hypothesis that the coefficient is 0 (thereby accepting that it is something other than 0). This doesn’t tell us that the numerical value of the coefficient we have is the correct value (hint: nothing tells us that!), but at least tells us that variable is significant enough to warrant inclusion in our linear model.

In our model the p-values for all eight of our predictors are small enough. The p-value for the intercept term is dubious, but it is a constant. We keep it in. The p-values for C6.Coarse.Aggregate and C7.Fine.Aggregate variables is a bit dodgy too…we stick with these for now.

Adding fitted values and prediction intervals

We can add the fitted values to our data set and also include the prediction interval. The prediction interval tells us that there is a 95% chance (according to our model, which could be wrong!) that the next data point corresponding with the given indepdendent variable values will lie in the given range.

We use the following command to create a new data frame that includes the prediction and intervals.

dat_pred=cbind(dat, predict(mfull,dat,interval="prediction"))

We can then explore the new data frame using head(dat_pred). Click on the top right arrow in below to navigate to the newly added columns of “fit”, which is the prediction, and the lower and upper prediction interval bounds lwr/upr.

We can explore the model fit visually by inspecting a series of plots that are available after the linear regression model is fit. For example, plot(mfull,1) produces a graph of the residuals vs the fitted values. Ideally, we are looking for a straight horizontal line around 0. This tests for non-linearity in the data and whether the variance of the residuals is constant (i.e. homoscedasticity). The way to test the former is to see a straight line and the latter to see the residuals randomly distributed above and below the line (and not increasing or decreasing with the fitted values). We don’t quite see a straight light (which tells us there may be non-linearity in how CCS varies with the independent variables), and do see some change in the variance of the residuals (more tightly spread at lower and higher fitted values). However, the problems are not severe enough to discard our linear modeling, but it tells us that we may want to explore a non-linear model at some point if higher accuracy is required.

The normal Q-Q plot shows whether the error (i.e. the residuals) are normally distributed. We want the plots to lie roughly on the line (which they do in this case).

A scale-location plot shows whether the residuals are evenly spread across the range of predicted values.

Cook’s distance helps to spot outliers. A data point having a large cook’s distance tells us that the data point has a strong influence on the predicted value. In the case below, data points 57, 225, and 611 have a large value.

In order to see the rows above, we can use the command: dat_pred[57,] (for 57th row):

The residuals vs. leverage plot helps us spot outliers, as well.

There are important future topics to consider, such as variable selection & transformations. It is very important with a method like linear regression to keep the model as simple as possible. If a variable is not increasing the performance of the model, leaving it in can cause problems such as overfit or making it more difficult to interpret the model.

Variable Selection

Our goal here is to try and fit a linear model given a set of variables. However, how do we determine whether a) we should use all or a subset of the variables, and b) to use the variables as-is or to transform them before use. We are not going to tackle transformations in this post, however we will talk about step-wise regression as one method to create simpler models and to decide whether a more complex model is worth it.

The following command fits models of increasing complexity (i.e. up to the full eight variables). The model measures errors for each model by running a 10-fold cross-validation (very useful as we avoid overfit from contaminating our results!).

This example code is from the following, more detailed post: http://www.sthda.com/english/articles/37-model-selection-essentials-in-r/154-stepwise-regression-essentials-in-r/

library(caret)
# Set seed for reproducibility
set.seed(123)
# Set up repeated k-fold cross-validation
train.control = trainControl(method = "cv", number = 10)
# Train the model
step.model = train(CCS ~., data = dat,
                    method = "leapBackward", 
                    tuneGrid = data.frame(nvmax = 1:8),
                    trControl = train.control
                    )
step.model$results

I can see from the table above that there isn’t a whole lot of difference in performance between the best 5-variable model and the best 8-variable model. Occam’s Razor* is a wonderful heuristic, and I’d take a simpler model over a more complicated one any day when it comes to a method like linear regression.

*More on Occam’s Razor: https://en.wikipedia.org/wiki/Occam%27s_razor

We can see which variables produce the best 1-variable model, 5-variable model, and so on, using:

summary(step.model$finalModel)
summary(step.model$finalModel)
Subset selection object
8 Variables  (and intercept)
                      Forced in Forced out
C1.Cement                 FALSE      FALSE
C2.Blast.Furnace.Slag     FALSE      FALSE
C3.Fly.Ash                FALSE      FALSE
C4.Water                  FALSE      FALSE
C5.Superplasticizer       FALSE      FALSE
C6.Coarse.Aggregate       FALSE      FALSE
C7.Fine.Aggregate         FALSE      FALSE
Age                       FALSE      FALSE
1 subsets of each size up to 6
Selection Algorithm: backward
         C1.Cement C2.Blast.Furnace.Slag C3.Fly.Ash C4.Water C5.Superplasticizer C6.Coarse.Aggregate
1  ( 1 ) "*"       " "                   " "        " "      " "                 " "                
2  ( 1 ) "*"       " "                   " "        " "      " "                 " "                
3  ( 1 ) "*"       " "                   " "        "*"      " "                 " "                
4  ( 1 ) "*"       "*"                   " "        "*"      " "                 " "                
5  ( 1 ) "*"       "*"                   "*"        "*"      " "                 " "                
6  ( 1 ) "*"       "*"                   "*"        "*"      "*"                 " "                
         C7.Fine.Aggregate Age
1  ( 1 ) " "               " "
2  ( 1 ) " "               "*"
3  ( 1 ) " "               "*"
4  ( 1 ) " "               "*"
5  ( 1 ) " "               "*"
6  ( 1 ) " "               "*"

In this case, the best 5-variable model includes the following independent variables: C1, C2, C3, C4, and Age. We discard C6 and C7 (which, if you recall, also had the dodgy p-values!).

OK - that’s enough linear regression for me for this weekend! Till next time!

LS0tDQpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sNCi0tLQ0KDQojIyBTb21lIHByZWxpbWluYXJpZXMNCg0KSWYgeW91IGhhdmUgbm90IGFscmVhZHksIHJlY29tbWVuZCByZXZpZXdpbmcgdGhlIGZvbGxvd2luZyBwb3N0cyBpbiBvcmRlciB0byByZXByb2R1Y2UgdGhlIHJlc3VsdHMgc2hvd24gYmVsb3cuDQoNCiAtIEluc3RhbGxpbmcgQW5hY29uZGEgKGh0dHBzOi8vcHJlZGljdGl2ZW1vZGVsZXIuY29tLzIwMTkvMDEvMTEvaW5zdGFsbGluZy1hbmFjb25kYS8pDQogLSBJbnN0YWxsaW5nIFJTdHVkaW8gKGh0dHBzOi8vcHJlZGljdGl2ZW1vZGVsZXIuY29tLzIwMTgvMTIvMDMvaW5zdGFsbGluZy1yc3R1ZGlvLykNCg0KVGhpcyBwb3N0IGlzIGNyZWF0ZWQgaW4gUlN0dWRpbyB1c2luZyBSIG5vdGVib29rIGZvcm1hdC4gRm9yIGFuIG92ZXJ2aWV3IG9mIFIgbm90ZWJvb2sgKm1hcmtkb3ducyosIGEgZ29vZCByZXNvdXJjZSBpczogaHR0cHM6Ly9ybWFya2Rvd24ucnN0dWRpby5jb20vYXV0aG9yaW5nX2Jhc2ljcy5odG1sIA0KDQojIyBCYWNrZ3JvdW5kDQoNCkluIHRoaXMgZXhhbXBsZSB3ZSBkZXZlbG9wIGEgYmFzaWMgcmVncmVzc2lvbiBtb2RlbCB0byBleHBsYWluIHRoZSB2YXJpYXRpb24gaW4gdGhlIGNvbXByZXNzaXZlIHN0cmVuZ3RoIG9mIGNvbmNyZXRlLiBDb25jcmV0ZSBpcyB0aGUgbGl0ZXJhbCBidWlsZGluZyBibG9jayB0aGF0IGhhcyBzaGFwZWQgbW9kZXJuIGxpdmluZy4gRmlndXJpbmcgb3V0IHdoYXQgbWFrZXMgaXQgc3Ryb25nZXIgaXMgYSB2ZXJ5IGltcG9ydGFudCB0b3BpYyBpbiBjb25zdHJ1Y3Rpb24uIA0KDQpXZSBoeXBvdGhlc2l6ZSB0aGF0IHRoZSBjb21wcmVzc2l2ZSBzdHJlbmd0aCBvZiBjb25jcmV0ZSBpcyBhIGZ1bmN0aW9uIG9mIHRoZSBhZ2UgYW5kIHRoZSBtaXh0dXJlIG9mIGluZ3JlZGllbnRzLiANCg0KVGhlIGRhdGEgdGhhdCB3ZSB1c2UgaW4gdGhpcyBleGFtcGxlIGNhbiBiZSBkb3dubG9hZGVkIGhlcmU6IGh0dHBzOi8vYXJjaGl2ZS5pY3MudWNpLmVkdS9tbC9kYXRhc2V0cy9Db25jcmV0ZStDb21wcmVzc2l2ZStTdHJlbmd0aA0KDQojIyBNb2RlbGluZw0KDQpUaGUgcHJvY2VzcyBvZiBsaW5lYXIgcmVncmVzc2lvbiBtb2RlbGluZyBicm9hZGx5IGludm9sdmVzIDEpIGV4cGxvcmluZyB0aGUgZGF0YSwgMikgdmFyaWFibGUgc2VsZWN0aW9uIDMpIGVzdGltYXRpbmcgdGhlIHBhcmFtZXRlcnMgb2YgdGhlIG1vZGVsLCBhbmQgNCkgZXZhbHVhdGluZyB0aGUgbW9kZWwuIEFuZCBub3QgbmVjZXNzYXJpbHkgaW4gdGhhdCBvcmRlciEgSSBpbGx1c3RyYXRlIGEgZmV3IG9mIHRoZXNlIHN0ZXBzIGJlbG93Lg0KDQojIyMgRGF0YSBFeHBsb3JhdGlvbg0KDQpGaXJzdCwgd2UgcmVhZCB0aGUgZGF0YSBpbi4gTm90aWNlIHRoZSAiZmlsZUVuY29kaW5nIiBhdHRyaWJ1dGUuIEkgaGFkIHRvIGluY2x1ZGUgaXQgYmVjYXVzZSB3ZSB3ZXJlIHJlYWRpbmcgaW4gYSBwZXNreSBoaWRkZW4gY2hhcmFjdGVyIGF0IHRoZSBzdGFydCBvZiB0aGUgZmlsZS4NCg0KYGBgDQpkYXQgPSByZWFkLmNzdigiRDovRGF0YS9QcmVkaWN0aXZlTW9kZWxlci9VQ0lfRGF0YS9Db25jcmV0ZUNvbXByZXNzaXZlL0NvbmNyZXRlX0RhdGEuY3N2IiwgaGVhZGVyID0gVFJVRSwgZmlsZUVuY29kaW5nPSJVVEYtOC1CT00iKQ0KYGBgDQpXZSBjYW4gdXNlIGhlYWQoZGF0KSBjb21tYW5kIHRvIGRpc3BsYXkgJiBpbnNwZWN0IHRoZSB0b3Agcm93cyBvZiB0aGUgZGF0YS4gWW91IGNhbiBjbGljayBvbiBidXR0b24gb24gdGhlIHRvcC1yaWdodCB0byBzaG93IG1vcmUgY29sdW1ucy4gVGhlIHZhcmlhYmxlIHRoYXQgd2Ugd2lzaCB0byBwcmVkaWN0L2VzdGltYXRlL2V4cGxhaW4gdGhlIHZhcmlhdGlvbiBpbiwgaXMgdGhlIENvbmNyZXRlIENvbXByZXNzaXZlIFN0cmVuZ3RoLCBvciAiQ0NTIiAoaS5lLiB0aGUgbGFzdCBjb2x1bW4pLg0KDQpgYGB7cn0NCmhlYWQoZGF0KQ0KYGBgDQoNCldlIGNoZWNrIHRoZSBudW1iZXIgb2Ygcm93cyBhbmQgY29sdW1ucyBpbiB0aGUgZGF0YSwgYXMgd2VsbCBhcyBob3cgdGhleSBhcmUgZGlzdHJpYnV0ZWQgdXNpbmcgdGhlIGNvbW1hbmRzIGJlbG93Lg0KDQpgYGB7cn0NCm5yb3coZGF0KSAjbnVtYmVyIG9mIHJvd3MNCm5jb2woZGF0KSAjbnVtYmVyIG9mIGNvbHVtbnMNCnN1bW1hcnkoZGF0KSAjbWluIC8gbWVhbiAvIG1lZGlhbiBldGMuDQpgYGANCg0KDQpPbmUgb2YgdGhlIGZpcnN0IHRoaW5ncyB0byBkbyBpcyB0byBzZWUgaG93IG91ciBkZXBlbmRlbnQgdmFyaWFibGUgKENDUykgaXMgZGlzdHJpYnV0ZWQuIFdlIGNhbiBjcmVhdGUgYSBxdWljayBoaXN0b2dyYW0gdXNpbmcgdGhlIGNvbW1hbmQ6IGhpc3QoZGF0JENDUykuIFRoaXMgaXNuJ3QgYSB0ZXN0IG9mIGFueXRoaW5nLCBidXQganVzdCBoZWxwcyB1cyB1bmRlcnN0YW5kIHRoZSByYW5nZSBvZiBvdXIgZGVwZW5kZW50IHZhbHVlcywgYW5kIGhvdyB0aG9zZSB2YWx1ZXMgYXJlIGRpc3RyaWJ1dGVkIGFsb25nIHRoYXQgcmFuZ2UuDQoNCmBgYHtSfQ0KaGlzdChkYXQkQ0NTKQ0KYGBgDQoNCk5leHQsIHdlIHdpc2ggdG8gc2VlIGhvdyBvdXIgKmluZGVwZW5kZW50KiB2YXJpYWJsZXMgYXJlIGNvcnJlbGF0ZWQgd2l0aCBvdXIgKmRlcGVuZGVudCogdmFyaWFibGUsIENDUy4gU3Ryb25nIHBvc2l0aXZlIG9yIG5lZ2F0aXZlIGNvcnJlbGF0aW9ucyB3aXRoIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgc3VnZ2VzdCB0aGF0IHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZSBtaWdodCBiZSBpbXBvcnRhbnQgaW4gdGVybXMgb2YgZXhwbGFpbmluZyB0aGUgZGVwZW5kZW50IHZhcmlhYmxlJ3MgdmFyaWF0aW9uLiBTdHJvbmcgY29ycmVsYXRpb25zIGFtb25nc3QgdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcyBzdWdnZXN0IHRoYXQgKm11bHRpY29sbGluZWFyaXR5KiAoTUMpIG1pZ2h0IGJlIGFuIGlzc3VlIGluIHRoZSBtb2RlbGluZy4gVGVjaG5pY2FsbHksIG11bHRpY29sbGluZWFyaXR5IGltcGxpZXMgdGhhdCBhbiBpbmRlcGVuZGVudCB2YXJpYWJsZSBtaWdodCBiZSBwcmVkaWN0ZWQgbGluZWFybHkgYnkgb25lIG9yIG1vcmUgb3RoZXIgaW5kZXBkZW50IHZhcmlhYmxlcyB0byBhIGhpZ2ggZGVncmVlIG9mIGFjY3VyYWN5LiBNQyBpcyBhbiBpc3N1ZSBiZWNhdXNlIGl0IG1ha2VzIHRoZSBjb2VmZmljaWVudCBlc3RpbWF0ZXMgaGlnaGx5IGRlcGVuZGVudCB1cG9uIHdoaWNoIHZhcmlhYmxlcyBhcmUgaW5jbHVkZWQgaW4gdGhlIG1vZGVsIGFuZCBtYWtlcyBpdCBkaWZmaWN1bHQgdG8gcHJvcGVybHkgaW50ZXJwcmV0IHRoZW0gKHdoaWxlIG5vdCBpbmNyZWFzaW5nIHRoZSBwcmVkaWN0aXZlIHBvd2VyIG9mIHRoZSBtb2RlbCBpbiBwcm9wb3J0aW9uIHRvIHRoZSBkaXNhZHZhbnRhZ2VzIGl0IGNhdXNlcyEpKi4gDQoNCldlIHVzZSBhIG5ldyBsaWJyYXJ5IGNhbGxlZCAicHN5Y2giIGluIG9yZGVyIHRvIGNyZWF0ZSB0aGVzZSBwbG90cywgYXMgYmVsb3cuIFRoZSBjb2RlIGNhbiBiZSBlZGl0ZWQgdG8gc2VlIGhvdyB0aGUgdmFyaWFibGVzIGFyZSBkaXN0cmlidXRlZCBhbmQgaG93IHRoZXkgYXJlIGNvcnJlbGF0ZWQgd2l0aCBlYWNoIG90aGVyLiBJZiB5b3UgZG8gbm90IGhhdmUgdGhlIGxpYnJhcnkgaW5zdGFsbGVkLCB5b3Ugd2lsbCBuZWVkIHRvIGRvIHNvIGJ5IGdvaW5nIHRvIFRvb2xzID4gSW5zdGFsbCBQYWNrYWdlcy4uLiBpbiB5b3VyIFJTdHVkaW8gcHJvZ3JhbS4NCg0KKkZvciBtb3JlIGluZm9ybWF0aW9uOiBodHRwczovL25ld29ubGluZWNvdXJzZXMuc2NpZW5jZS5wc3UuZWR1L3N0YXQ1MDEvbm9kZS8zNDYvIA0KDQpgYGANCmxpYnJhcnkocHN5Y2gpDQpwYWlycy5wYW5lbHMoZGF0W2MoOSwxLDc6OCldLCANCiAgICAgICAgICAgICBtZXRob2QgPSAicGVhcnNvbiIsICMgY29ycmVsYXRpb24gbWV0aG9kDQogICAgICAgICAgICAgaGlzdC5jb2wgPSAiIzAwQUZCQiIsDQogICAgICAgICAgICAgZGVuc2l0eSA9IFRSVUUsICAjIHNob3cgZGVuc2l0eSBwbG90cw0KICAgICAgICAgICAgIGVsbGlwc2VzID0gVFJVRSAjIHNob3cgY29ycmVsYXRpb24gZWxsaXBzZXMNCiAgICAgICAgICAgICApDQpgYGANCg0KYGBge1J9DQpsaWJyYXJ5KHBzeWNoKQ0KcGFpcnMucGFuZWxzKGRhdFtjKDksMSw3OjgpXSwgDQogICAgICAgICAgICAgbWV0aG9kID0gInBlYXJzb24iLCAjIGNvcnJlbGF0aW9uIG1ldGhvZA0KICAgICAgICAgICAgIGhpc3QuY29sID0gIiMwMEFGQkIiLA0KICAgICAgICAgICAgIGRlbnNpdHkgPSBUUlVFLCAgIyBzaG93IGRlbnNpdHkgcGxvdHMNCiAgICAgICAgICAgICBlbGxpcHNlcyA9IFRSVUUgIyBzaG93IGNvcnJlbGF0aW9uIGVsbGlwc2VzDQogICAgICAgICAgICAgKQ0KYGBgDQoNCg0KIyMjIEtleSBBc3N1bXB0aW9uczogTGluZWFyIFJlZ3Jlc3Npb24NCg0KVGhlIGtleSBhc3N1bXB0aW9ucyBvZiBsaW5lYXIgcmVncmVzc2lvbiB0aGF0IHdlIHdhbnQgdG8gdGVzdCBpbmNsdWRlOg0KDQoxLiBMaW5lYXJpdHkgJiBBZGRpdGl2aXR5OiBBbiBhc3N1bXB0aW9uIHRoYXQgdGhlIGRlcGVuZGVudCB2YXJpYWJsZSBpcyBhIGxpbmVhciBhbmQgYWRkaXRpdmUgZnVuY3Rpb24gb2YgdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcy4NCjIuIEluZGVwZW5kZW5jZSBvZiBlcnJvcnMgKHJlc2lkdWFscyk6IFRoZSBlcnJvcnMgYXJlIG5vdCBjb3JyZWxhdGVkLg0KMy4gQ29uc3RhbnQgdmFyaWFuY2Ugb2YgZXJyb3JzIChhbHNvIGNhbGxlZCBieSBhIG5lZWRsZXNzbHkgY29tcGxpY2F0ZWQgbmFtZTogaG9tb3NjZWRhc3RpY2l0eSk6IFRoZSB2YXJpYW5jZSBvZiB0aGUgcmVzaWR1YWxzIHNob3VsZCBiZSBhcHByb3hpbWF0ZWx5IGVxdWFsIGFjcm9zcyB0aGUgcmFuZ2Ugb2YgdGhlIGRlcGVuZGVudCB2YXJpYWJsZSAoQ0NTIGluIG91ciBjYXNlKQ0KNC4gRXJyb3JzIGFyZSBkaXN0cmlidXRlZCBub3JtYWxseTogR2VuZXJhbGx5IGxlc3MgaW1wb3J0YW50IHRoYW4gdGhlIGFzc3VtcHRpb25zIGFib3ZlLCB3ZSBpZGVhbGx5IHdhbnQgdG8gc2VlIHRoZSByZXNpZHVhbHMgZGlzdHJpYnR1ZWQgbm9ybWFsbHkuDQoNCiMjIyBGdW5jdGlvbmFsIEZvcm0NCg0KV2UgYXJlIHNraXBwaW5nIHRoaXMgc3RlcCBoZXJlLCBhcyB3ZSBhcmUgKmFzc3VtaW5nKiBhICpsaW5lYXIqIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZXMgYW5kIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUuIEkgd2lsbCBtYWtlIGEgcG9zdCBsYXRlciBvbiBob3cgdG8gZXN0aW1hdGUgYSBmdW5jdGlvbmFsIGZvcm0gZ2l2ZW4gZGF0YS4NCg0KDQojIyMgUGFyYW1ldGVyIEVzdGltYXRpb24NCg0KTmV4dCwgd2UgZml0IGEgbGluZWFyIG1vZGVsIHVzaW5nIGFsbCBvZiB0aGUgKmluZGVwZW5kZW50KiB2YXJpYWJsZXMgdGhhdCB3ZSBoYXZlLiBXZSBjYW4gZG8gc28gdXNpbmcgdGhlIGZvbGxvd2luZyBjb21tYW5kOg0KDQpgYGANCm1mdWxsPWxtKENDU34uLGRhdGE9ZGF0KQ0KYGBgDQoNCldlIGNhbGwgb3VyIG1vZGVsICJtZnVsbCIgKG9yIGZ1bGwgbW9kZWwpLiBXZSByZWdyZXNzIHRoZSAqZGVwZW5kZW50KiB2YXJpYWJsZSBDQ1Mgb24gYWxsIG90aGVyIHZhcmlhYmxlcyB1c2luZyB0aGUgIn4uIiBjb21tYW5kLiBXZSBjYW4gYWxzbyBzZWxlY3QgY2VydGFpbiB2YXJpYWJsZXMgdXNpbmcgIkNDU35BZ2UrQzEuQ2VtZW50IiBldGMuIA0KDQpOZXh0LCB3ZSBkaXNwbGF5IGEgc3VtbWFyeSBvZiB0aGUgZml0dGVkIG1vZGVsLg0KDQpgYGB7cn0NCnN1bW1hcnkobWZ1bGwpDQpgYGANCg0KSGVyZSBhcmUgYSBmZXcga2V5IG9ic2VydmF0aW9ucyBmcm9tIHRoZSBsaW5lYXIgcmVncmVzc2lvbiBzdW1tYXJ5Og0KDQojIyMjIFItU3F1YXJlIA0KDQpXZSBoYXZlIGFuIFItc3F1YXJlIG9mIDYxLjU1JSwgYW5kIGFuIGFkanVzdGVkIFItU3F1YXJlIG9mIGFib3V0IHRoZSBzYW1lLiBUaGlzIGlzIGdvb2QsIGJlY2F1c2UgdGhlIGFkanVzdGVkIFItU3F1YXJlICphZGp1c3RzKiB0aGUgbWV0cmljIGZvciB0aGUgbnVtYmVyIG9mIHByZWRpY3RvcnMgaW4gdGhlIG1vZGVsLiBUaGUgYWRqdXN0ZWQgUi1TcXVhcmUgaXMgYWx3YXlzIGxvd2VyIHRoYW4gUi1TcXVhcmUsIGFuZCBhIGJpZyBkaWZmZXJlbmNlIGhpbnRzIHRoYXQgdGhlIG1vZGVsIG1pZ2h0IGhhdmUgdG9vIG1hbnkgcHJlZGljdG9ycyB0aGFuIGlzIHJlcXVpcmVkLCBhbmQgdGhlIGZvbGxvd2luZyBzZWN0aW9uIG9uIHZhcmlhYmxlIHNlbGVjdGlvbiBiZWNvbWVzIG1vcmUgaW1wb3J0YW50IQ0KDQpNb3JlIGluZm9ybWF0aW9uOiANCg0KKiBodHRwOi8vYmxvZy5taW5pdGFiLmNvbS9ibG9nL2FkdmVudHVyZXMtaW4tc3RhdGlzdGljcy0yL211bHRpcGxlLXJlZ2Vzc2lvbi1hbmFseXNpcy11c2UtYWRqdXN0ZWQtci1zcXVhcmVkLWFuZC1wcmVkaWN0ZWQtci1zcXVhcmVkLXRvLWluY2x1ZGUtdGhlLWNvcnJlY3QtbnVtYmVyLW9mLXZhcmlhYmxlcw0KKiBodHRwOi8vdGhlc3RhdHNnZWVrLmNvbS8yMDEzLzEwLzI4L3Itc3F1YXJlZC1hbmQtYWRqdXN0ZWQtci1zcXVhcmVkLw0KDQoNCiMjIyMgT3ZlcmFsbCBGLXN0YXRpc3RpYyAvIHAtdmFsdWUNCg0KVGhlIEYtdGVzdCBpbiBhIG11bHRpdmFyaWFibGUgbGluZWFyIHJlZ3Jlc3Npb24gaGVscHMgdXMgZGV0ZXJtaW5lIHdoZXRoZXIgKmFueSogb2YgdGhlIGluZGVwZW5kZW50IHZhcmlhYmxlcyBpcyBzaWduaWZpY2FudC4gTm90ZSB0aGF0IGl0IGRvZXMgbm90IHRlbGwgdXMgdGhhdCAqYWxsKiBvciAqbW9zdCogb2YgdGhlIHZhcmlhYmxlcyBhcmUgc2lnbmlmaWNhbnQsIGZvciB0aGF0IHdlIGhhdmUgdG8gaW5zcGVjdCBvdGhlciBkaWFnbm9zdGljcyAoc2VlIGJlbG93KS4NCg0KVGhlIG92ZXJhbGwgRi1zdGF0IHByb2R1Y2VzIGEgdmVyeSB0aW55IHAtdmFsdWUsIHNvIHdlIGNhbiByZWplY3QgdGhlICpudWxsIGh5cG90aGVzZXMqIHRoYXQgYWxsIG9mIHRoZSBjb2VmZmljaWVudHMgYXJlIGluIGZhY3QgMCAoaS5lLiBubyB2YXJpYWJsZSBpcyBzaWduaWZpY2FudCkuIEdlbmVyYWxseSBzcGVha2luZywgd2Ugd2FudCB0aGlzIHZhbHVlIHRvIGJlIGJlbG93IDAuMDUuDQoNCk1vcmUgaW5mb3JtYXRpb246IA0KDQoqIGh0dHA6Ly9mYWN3ZWIuY3MuZGVwYXVsLmVkdS9zam9zdC9jc2M0MjMvZG9jdW1lbnRzL2YtdGVzdC1yZWcuaHRtDQoNCiMjIyMgUmVzaWR1YWwgU3RhbmRhcmQgRXJyb3INCg0KVGhlIG1vZGVsIGhhcyBhIFJlc2lkdWFsIFN0YW5kYXJkIEVycm9yIChSU0UpIG9mIDEwLjQuIFRoaXMsIGFsb25nIHdpdGggdGhlIFItc3F1YXJlLCBpcyBhIG1lYXN1cmUgb2YgdGhlICpnb29kbmVzcyBvZiBmaXQqIG9mIHRoZSBtb2RlbC5UaGUgU3RhbmRhcmQgRXJyb3IgKFNFKSB0ZWxscyB1cyB0aGUgYXZlcmFnZSBlcnJvciBiZXR3ZWVuIHRoZSBkYXRhIHBvaW50cyBhbmQgdGhlIGZpdHRlZCBsaW5lLiBJdCBpcyBjYWxjdWxhdGVkIGFzOg0KDQokJA0KUlNFID0gXGhhdHtcc2lnbWF9ID0gXHNxcnR7XGZyYWN7XFNpZ21hX3tpPTF9XntufXsoWV9pIC1caGF0e1lfaX0pXjJ9fSB7bi1rLTF9IH0NCiQkDQpXaGVyZSAkayQgaXMgdGhlIG51bWJlciBvZiBwcmVkaWN0b3JzIGluIHRoZSBtb2RlbCBhbmQgJG4kIGlzIHRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zLiBJbiB0aGUgY2FzZSBvZiBvdXIgbW9kZWwsIGFuIFJTRSBvZiAxMC40IGlzIHRlbGxpbmcgdXMgdGhhdCBvbiBhdmVyYWdlIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIHJlcG9ydGVkIGRhdGEgYW5kIGZpdHRlZCBsaW5lIGlzIDEwLjQuIElmIHRoZSByZXNpZHVhbHMgYXJlIGRpc3RyaWJ1dGVkIG5vcm1hbGx5LCB0aGVuIGFib3V0IHR3by10aGlyZHMgb2YgdGhlIGVycm9ycyBhcmUgaW4gdGhlICRccG0xMC40JCByYW5nZSBhbmQgYWJvdXQgOTUlIG9mIHRoZSBlcnJvcnMgYXJlIGluIHRoZSAkXHBtMjAuOCQgcmFuZ2UuDQoNCiMjIyMgVmFyaWFibGUgc2lnbmlmaWNhbmNlIHRlc3RzDQoNCkluIHRoZSBzdW1tYXJ5IG91dHB1dCBhYm92ZSwgd2UgaGF2ZSBmb3IgZWFjaCB2YXJpYWJsZTogMSkgdGhlIGNvZWZmaWNpZW50IGVzdGltYXRlLCAyKSB0aGUgc3RhbmRhcmQgZXJyb3IsIDMpIHRoZSB0LXZhbHVlLCBhbmQgNCkgdGhlIHAtdmFsdWUuIElmIHRoZSBwLXZhbHVlIGlzIHNtYWxsIGVub3VnaCAoZS5nLiBsZXNzIHRoYW4gMC4wNSksIHRoZW4gd2UgdGVuZCB0byByZWplY3QgdGhlICpudWxsKiBoeXBvdGhlc2lzIHRoYXQgdGhlIGNvZWZmaWNpZW50IGlzIDAgKHRoZXJlYnkgYWNjZXB0aW5nIHRoYXQgaXQgaXMgc29tZXRoaW5nIG90aGVyIHRoYW4gMCkuIFRoaXMgZG9lc24ndCB0ZWxsIHVzIHRoYXQgdGhlIG51bWVyaWNhbCB2YWx1ZSBvZiB0aGUgY29lZmZpY2llbnQgd2UgaGF2ZSBpcyAqKnRoZSBjb3JyZWN0KiogdmFsdWUgKGhpbnQ6IG5vdGhpbmcgdGVsbHMgdXMgdGhhdCEpLCBidXQgYXQgbGVhc3QgdGVsbHMgdXMgdGhhdCB2YXJpYWJsZSBpcyBzaWduaWZpY2FudCBlbm91Z2ggdG8gd2FycmFudCBpbmNsdXNpb24gaW4gb3VyIGxpbmVhciBtb2RlbC4NCg0KSW4gb3VyIG1vZGVsIHRoZSBwLXZhbHVlcyBmb3IgYWxsIGVpZ2h0IG9mIG91ciBwcmVkaWN0b3JzIGFyZSBzbWFsbCBlbm91Z2guIFRoZSBwLXZhbHVlIGZvciB0aGUgaW50ZXJjZXB0IHRlcm0gaXMgZHViaW91cywgYnV0IGl0IGlzIGEgY29uc3RhbnQuIFdlIGtlZXAgaXQgaW4uIFRoZSBwLXZhbHVlcyBmb3IgQzYuQ29hcnNlLkFnZ3JlZ2F0ZSBhbmQgQzcuRmluZS5BZ2dyZWdhdGUgdmFyaWFibGVzIGlzIGEgYml0IGRvZGd5IHRvby4uLndlIHN0aWNrIHdpdGggdGhlc2UgZm9yIG5vdy4NCg0KIyMjIyBBZGRpbmcgZml0dGVkIHZhbHVlcyBhbmQgcHJlZGljdGlvbiBpbnRlcnZhbHMNCg0KV2UgY2FuIGFkZCB0aGUgZml0dGVkIHZhbHVlcyB0byBvdXIgZGF0YSBzZXQgYW5kIGFsc28gaW5jbHVkZSB0aGUgcHJlZGljdGlvbiBpbnRlcnZhbC4gVGhlIHByZWRpY3Rpb24gaW50ZXJ2YWwgdGVsbHMgdXMgdGhhdCB0aGVyZSBpcyBhIDk1JSBjaGFuY2UgKGFjY29yZGluZyB0byBvdXIgbW9kZWwsIHdoaWNoIGNvdWxkIGJlIHdyb25nISkgdGhhdCB0aGUgbmV4dCBkYXRhIHBvaW50IGNvcnJlc3BvbmRpbmcgd2l0aCB0aGUgZ2l2ZW4gaW5kZXBkZW5kZW50IHZhcmlhYmxlIHZhbHVlcyB3aWxsIGxpZSBpbiB0aGUgZ2l2ZW4gcmFuZ2UuDQoNCldlIHVzZSB0aGUgZm9sbG93aW5nIGNvbW1hbmQgdG8gY3JlYXRlIGEgbmV3IGRhdGEgZnJhbWUgdGhhdCBpbmNsdWRlcyB0aGUgcHJlZGljdGlvbiBhbmQgaW50ZXJ2YWxzLg0KDQpgYGANCmRhdF9wcmVkPWNiaW5kKGRhdCwgcHJlZGljdChtZnVsbCxkYXQsaW50ZXJ2YWw9InByZWRpY3Rpb24iKSkNCmBgYA0KV2UgY2FuIHRoZW4gZXhwbG9yZSB0aGUgbmV3IGRhdGEgZnJhbWUgdXNpbmcgaGVhZChkYXRfcHJlZCkuIENsaWNrIG9uIHRoZSB0b3AgcmlnaHQgYXJyb3cgaW4gYmVsb3cgdG8gbmF2aWdhdGUgdG8gdGhlIG5ld2x5IGFkZGVkIGNvbHVtbnMgb2YgImZpdCIsIHdoaWNoIGlzIHRoZSBwcmVkaWN0aW9uLCBhbmQgdGhlIGxvd2VyIGFuZCB1cHBlciBwcmVkaWN0aW9uIGludGVydmFsIGJvdW5kcyBsd3IvdXByLg0KDQpgYGB7cn0NCmhlYWQoZGF0X3ByZWQpDQoNCmBgYA0KDQpXZSBjYW4gZXhwbG9yZSB0aGUgbW9kZWwgZml0IHZpc3VhbGx5IGJ5IGluc3BlY3RpbmcgYSBzZXJpZXMgb2YgcGxvdHMgdGhhdCBhcmUgYXZhaWxhYmxlIGFmdGVyIHRoZSBsaW5lYXIgcmVncmVzc2lvbiBtb2RlbCBpcyBmaXQuIEZvciBleGFtcGxlLCBwbG90KG1mdWxsLDEpIHByb2R1Y2VzIGEgZ3JhcGggb2YgdGhlIHJlc2lkdWFscyB2cyB0aGUgZml0dGVkIHZhbHVlcy4gSWRlYWxseSwgd2UgYXJlIGxvb2tpbmcgZm9yIGEgc3RyYWlnaHQgaG9yaXpvbnRhbCBsaW5lIGFyb3VuZCAwLiBUaGlzIHRlc3RzIGZvciBub24tbGluZWFyaXR5IGluIHRoZSBkYXRhIGFuZCB3aGV0aGVyIHRoZSB2YXJpYW5jZSBvZiB0aGUgcmVzaWR1YWxzIGlzIGNvbnN0YW50IChpLmUuIGhvbW9zY2VkYXN0aWNpdHkpLiBUaGUgd2F5IHRvIHRlc3QgdGhlIGZvcm1lciBpcyB0byBzZWUgYSBzdHJhaWdodCBsaW5lIGFuZCB0aGUgbGF0dGVyIHRvIHNlZSB0aGUgcmVzaWR1YWxzIHJhbmRvbWx5IGRpc3RyaWJ1dGVkIGFib3ZlIGFuZCBiZWxvdyB0aGUgbGluZSAoYW5kIG5vdCBpbmNyZWFzaW5nIG9yIGRlY3JlYXNpbmcgd2l0aCB0aGUgZml0dGVkIHZhbHVlcykuIFdlIGRvbid0IHF1aXRlIHNlZSBhIHN0cmFpZ2h0IGxpZ2h0ICh3aGljaCB0ZWxscyB1cyB0aGVyZSBtYXkgYmUgbm9uLWxpbmVhcml0eSBpbiBob3cgQ0NTIHZhcmllcyB3aXRoIHRoZSBpbmRlcGVuZGVudCB2YXJpYWJsZXMpLCBhbmQgZG8gc2VlIHNvbWUgY2hhbmdlIGluIHRoZSB2YXJpYW5jZSBvZiB0aGUgcmVzaWR1YWxzIChtb3JlIHRpZ2h0bHkgc3ByZWFkIGF0IGxvd2VyIGFuZCBoaWdoZXIgZml0dGVkIHZhbHVlcykuIEhvd2V2ZXIsIHRoZSBwcm9ibGVtcyBhcmUgbm90IHNldmVyZSBlbm91Z2ggdG8gZGlzY2FyZCBvdXIgbGluZWFyIG1vZGVsaW5nLCBidXQgaXQgdGVsbHMgdXMgdGhhdCB3ZSBtYXkgd2FudCB0byBleHBsb3JlIGEgbm9uLWxpbmVhciBtb2RlbCBhdCBzb21lIHBvaW50IGlmIGhpZ2hlciBhY2N1cmFjeSBpcyByZXF1aXJlZC4NCg0KYGBge3J9DQpwbG90KG1mdWxsLDEpDQpgYGANCg0KVGhlIG5vcm1hbCBRLVEgcGxvdCBzaG93cyB3aGV0aGVyIHRoZSBlcnJvciAoaS5lLiB0aGUgKnJlc2lkdWFscyopIGFyZSBub3JtYWxseSBkaXN0cmlidXRlZC4gV2Ugd2FudCB0aGUgcGxvdHMgdG8gbGllIHJvdWdobHkgb24gdGhlIGxpbmUgKHdoaWNoIHRoZXkgZG8gaW4gdGhpcyBjYXNlKS4NCg0KYGBge3J9DQpwbG90KG1mdWxsLDIpDQpgYGANCg0KQSBzY2FsZS1sb2NhdGlvbiBwbG90IHNob3dzIHdoZXRoZXIgdGhlIHJlc2lkdWFscyBhcmUgZXZlbmx5IHNwcmVhZCBhY3Jvc3MgdGhlIHJhbmdlIG9mIHByZWRpY3RlZCB2YWx1ZXMuDQoNCmBgYHtyfQ0KcGxvdChtZnVsbCwzKQ0KYGBgDQoNCkNvb2sncyBkaXN0YW5jZSBoZWxwcyB0byBzcG90IG91dGxpZXJzLiBBIGRhdGEgcG9pbnQgaGF2aW5nIGEgbGFyZ2UgY29vaydzIGRpc3RhbmNlIHRlbGxzIHVzIHRoYXQgdGhlIGRhdGEgcG9pbnQgaGFzIGEgc3Ryb25nIGluZmx1ZW5jZSBvbiB0aGUgcHJlZGljdGVkIHZhbHVlLiBJbiB0aGUgY2FzZSBiZWxvdywgZGF0YSBwb2ludHMgNTcsIDIyNSwgYW5kIDYxMSBoYXZlIGEgbGFyZ2UgdmFsdWUuIA0KDQpgYGB7cn0NCnBsb3QobWZ1bGwsNCkNCmBgYA0KDQpJbiBvcmRlciB0byBzZWUgdGhlIHJvd3MgYWJvdmUsIHdlIGNhbiB1c2UgdGhlIGNvbW1hbmQ6IGRhdF9wcmVkWzU3LF0gKGZvciA1N3RoIHJvdyk6IA0KDQpgYGB7cn0NCmRhdF9wcmVkWzU3LF0NCmBgYA0KDQoNClRoZSByZXNpZHVhbHMgdnMuIGxldmVyYWdlIHBsb3QgaGVscHMgdXMgc3BvdCBvdXRsaWVycywgYXMgd2VsbC4NCg0KYGBge3J9DQpwbG90KG1mdWxsLDUpDQpgYGANCg0KVGhlcmUgYXJlIGltcG9ydGFudCBmdXR1cmUgdG9waWNzIHRvIGNvbnNpZGVyLCBzdWNoIGFzIHZhcmlhYmxlIHNlbGVjdGlvbiAmIHRyYW5zZm9ybWF0aW9ucy4gSXQgaXMgdmVyeSBpbXBvcnRhbnQgd2l0aCBhIG1ldGhvZCBsaWtlIGxpbmVhciByZWdyZXNzaW9uIHRvIGtlZXAgdGhlIG1vZGVsIGFzIHNpbXBsZSBhcyBwb3NzaWJsZS4gSWYgYSB2YXJpYWJsZSBpcyBub3QgaW5jcmVhc2luZyB0aGUgcGVyZm9ybWFuY2Ugb2YgdGhlIG1vZGVsLCBsZWF2aW5nIGl0IGluIGNhbiBjYXVzZSBwcm9ibGVtcyBzdWNoIGFzIG92ZXJmaXQgb3IgbWFraW5nIGl0IG1vcmUgZGlmZmljdWx0IHRvIGludGVycHJldCB0aGUgbW9kZWwuIA0KDQojIyMgVmFyaWFibGUgU2VsZWN0aW9uDQoNCk91ciBnb2FsIGhlcmUgaXMgdG8gdHJ5IGFuZCBmaXQgYSBsaW5lYXIgbW9kZWwgZ2l2ZW4gYSBzZXQgb2YgdmFyaWFibGVzLiBIb3dldmVyLCBob3cgZG8gd2UgZGV0ZXJtaW5lIHdoZXRoZXIgYSkgd2Ugc2hvdWxkIHVzZSBhbGwgb3IgYSBzdWJzZXQgb2YgdGhlIHZhcmlhYmxlcywgYW5kIGIpIHRvIHVzZSB0aGUgdmFyaWFibGVzIGFzLWlzIG9yIHRvIHRyYW5zZm9ybSB0aGVtIGJlZm9yZSB1c2UuIFdlIGFyZSBub3QgZ29pbmcgdG8gdGFja2xlIHRyYW5zZm9ybWF0aW9ucyBpbiB0aGlzIHBvc3QsIGhvd2V2ZXIgd2Ugd2lsbCB0YWxrIGFib3V0IHN0ZXAtd2lzZSByZWdyZXNzaW9uIGFzIG9uZSBtZXRob2QgdG8gY3JlYXRlIHNpbXBsZXIgbW9kZWxzIGFuZCB0byBkZWNpZGUgd2hldGhlciBhIG1vcmUgY29tcGxleCBtb2RlbCBpcyB3b3J0aCBpdC4NCg0KVGhlIGZvbGxvd2luZyBjb21tYW5kIGZpdHMgbW9kZWxzIG9mIGluY3JlYXNpbmcgY29tcGxleGl0eSAoaS5lLiB1cCB0byB0aGUgZnVsbCBlaWdodCB2YXJpYWJsZXMpLiBUaGUgbW9kZWwgbWVhc3VyZXMgZXJyb3JzIGZvciBlYWNoIG1vZGVsIGJ5IHJ1bm5pbmcgYSAxMC1mb2xkIGNyb3NzLXZhbGlkYXRpb24gKHZlcnkgdXNlZnVsIGFzIHdlIGF2b2lkIG92ZXJmaXQgZnJvbSBjb250YW1pbmF0aW5nIG91ciByZXN1bHRzISkuIA0KDQpUaGlzIGV4YW1wbGUgY29kZSBpcyBmcm9tIHRoZSBmb2xsb3dpbmcsIG1vcmUgZGV0YWlsZWQgcG9zdDogaHR0cDovL3d3dy5zdGhkYS5jb20vZW5nbGlzaC9hcnRpY2xlcy8zNy1tb2RlbC1zZWxlY3Rpb24tZXNzZW50aWFscy1pbi1yLzE1NC1zdGVwd2lzZS1yZWdyZXNzaW9uLWVzc2VudGlhbHMtaW4tci8NCg0KYGBgDQpsaWJyYXJ5KGNhcmV0KQ0KIyBTZXQgc2VlZCBmb3IgcmVwcm9kdWNpYmlsaXR5DQpzZXQuc2VlZCgxMjMpDQojIFNldCB1cCByZXBlYXRlZCBrLWZvbGQgY3Jvc3MtdmFsaWRhdGlvbg0KdHJhaW4uY29udHJvbCA9IHRyYWluQ29udHJvbChtZXRob2QgPSAiY3YiLCBudW1iZXIgPSAxMCkNCiMgVHJhaW4gdGhlIG1vZGVsDQpzdGVwLm1vZGVsID0gdHJhaW4oQ0NTIH4uLCBkYXRhID0gZGF0LA0KICAgICAgICAgICAgICAgICAgICBtZXRob2QgPSAibGVhcEJhY2t3YXJkIiwgDQogICAgICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gZGF0YS5mcmFtZShudm1heCA9IDE6OCksDQogICAgICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IHRyYWluLmNvbnRyb2wNCiAgICAgICAgICAgICAgICAgICAgKQ0Kc3RlcC5tb2RlbCRyZXN1bHRzDQpgYGANCg0KYGBge3J9DQpsaWJyYXJ5KGNhcmV0KQ0KIyBTZXQgc2VlZCBmb3IgcmVwcm9kdWNpYmlsaXR5DQpzZXQuc2VlZCgxMjMpDQojIFNldCB1cCByZXBlYXRlZCBrLWZvbGQgY3Jvc3MtdmFsaWRhdGlvbg0KdHJhaW4uY29udHJvbCA9IHRyYWluQ29udHJvbChtZXRob2QgPSAiY3YiLCBudW1iZXIgPSAxMCkNCiMgVHJhaW4gdGhlIG1vZGVsDQpzdGVwLm1vZGVsID0gdHJhaW4oQ0NTIH4uLCBkYXRhID0gZGF0LA0KICAgICAgICAgICAgICAgICAgICBtZXRob2QgPSAibGVhcEJhY2t3YXJkIiwgDQogICAgICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gZGF0YS5mcmFtZShudm1heCA9IDE6OCksDQogICAgICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IHRyYWluLmNvbnRyb2wNCiAgICAgICAgICAgICAgICAgICAgKQ0Kc3RlcC5tb2RlbCRyZXN1bHRzDQpgYGANCg0KSSBjYW4gc2VlIGZyb20gdGhlIHRhYmxlIGFib3ZlIHRoYXQgdGhlcmUgaXNuJ3QgYSB3aG9sZSBsb3Qgb2YgZGlmZmVyZW5jZSBpbiBwZXJmb3JtYW5jZSBiZXR3ZWVuIHRoZSBiZXN0IDUtdmFyaWFibGUgbW9kZWwgYW5kIHRoZSBiZXN0IDgtdmFyaWFibGUgbW9kZWwuIE9jY2FtJ3MgUmF6b3IqIGlzIGEgd29uZGVyZnVsIGhldXJpc3RpYywgYW5kIEknZCB0YWtlIGEgc2ltcGxlciBtb2RlbCBvdmVyIGEgbW9yZSBjb21wbGljYXRlZCBvbmUgYW55IGRheSB3aGVuIGl0IGNvbWVzIHRvIGEgbWV0aG9kIGxpa2UgbGluZWFyIHJlZ3Jlc3Npb24uIA0KDQoqTW9yZSBvbiBPY2NhbSdzIFJhem9yOiBodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9PY2NhbSUyN3NfcmF6b3IgDQoNCldlIGNhbiBzZWUgd2hpY2ggdmFyaWFibGVzIHByb2R1Y2UgdGhlIGJlc3QgMS12YXJpYWJsZSBtb2RlbCwgNS12YXJpYWJsZSBtb2RlbCwgYW5kIHNvIG9uLCB1c2luZzoNCg0KYGBgDQpzdW1tYXJ5KHN0ZXAubW9kZWwkZmluYWxNb2RlbCkNCmBgYA0KDQpgYGB7Un0NCnN1bW1hcnkoc3RlcC5tb2RlbCRmaW5hbE1vZGVsKQ0KYGBgDQoNCkluIHRoaXMgY2FzZSwgdGhlIGJlc3QgNS12YXJpYWJsZSBtb2RlbCBpbmNsdWRlcyB0aGUgZm9sbG93aW5nIGluZGVwZW5kZW50IHZhcmlhYmxlczogQzEsIEMyLCBDMywgQzQsIGFuZCBBZ2UuIFdlIGRpc2NhcmQgQzYgYW5kIEM3ICh3aGljaCwgaWYgeW91IHJlY2FsbCwgYWxzbyBoYWQgdGhlIGRvZGd5IHAtdmFsdWVzISkuDQoNCk9LIC0gdGhhdCdzIGVub3VnaCBsaW5lYXIgcmVncmVzc2lvbiBmb3IgbWUgZm9yIHRoaXMgd2Vla2VuZCEgVGlsbCBuZXh0IHRpbWUh