If you want to compare the execution time between 2 functions, the easiest method to measure the time is timeit.timeit. You can also specify the number of times you want to rerun your function to get a better estimation …
Compare the execution time between 2 functions Read More »
Have you ever wanted to use dates in a month as the feature in your time series data? You can find the days in a month by using calendar.monthrange(year, month)[1] like above.
Use Dates in a Month as the Feature Read More »
Have you wanted to extract series data from various Internet sources directly into a pandas DataFrame? That is when pandas_reader comes in handy.
Above is the snippet to extract daily data of AD indicator from 2008 to 2018. All you …
Extract series data from various Internet sources directly into a pandas DataFrame Read More »
If you want to assign values to multiple new columns, instead of assigning them separately, you can do everything in one line of code with df.assign.
In the code above, I first created col3 then use col3 to create …
Assign Values to Multiple New Columns Read More »
If you want to reduce the memory of your pandas DataFrame, start with changing the data type of a column. If your categorical variable has low cardinality, change the data type to category like below.
The memory is now is …
Reduce pd.DataFrame’s Memory Read More »
How do you use cross-validation when working with time series? Since the dates of test data must be later than the dates of train data, you wouldn’t want to shuffle in cross validator. The best way is to use TimeSeriesSplit…
Cross Validation with Time Series Read More »
Outliers are unusual values in your dataset, and they can distort statistical analyses. If you want to trim values that the outliers, one of the methods is to use df.clip.
Above is how to use the 0.5-quantile as the …
If you want to create a cute bar chart for a representation, check out cutecharts. Here is an example of how to create one.
…
Sometimes it is important to compare the similar features between 2 different datasets side by side such as comparing train and test sets. If you want to quickly compare 2 datasets through graphs, check out sweetviz.
Run the code above …
Compare the similar features between 2 different datasets Read More »
If you want to get all data starting in a particular year and exclude the previous years, simply use df.loc['year':] like above. This works when the index of your pd.Dataframe is DatetimeIndex.
Pandas DataFrame Get Data in a Year Range Read More »
