- Python
- XPath
- Web scraping
- Visualization
- Tidy data
Keywords
About the U.S. minimum wage
http://www.dol.gov/whd/minwage/america.htm
Discovered this historical data:
http://www.dol.gov/whd/state/stateMinWageHis.htm
About “tidy data”
A huge amount of effort is spent cleaning data to get it ready for analysis, but there has been little research on how to make data cleaning as easy and effective as possible. This paper tackles a small, but important, component of data cleaning: data tidying. Tidy datasets are easy to manipulate, model and visualize, and have a specific structure: each variable is a column, each observation is a row, and each type of observational unit is a table. This framework makes it easy to tidy messy datasets because only a small set of tools are needed to deal with a wide range of un-tidy datasets. This structure also makes it easier to develop tidy tools for data analysis, tools that both input and output tidy datasets. The advantages of a consistent data structure and matching tools are demonstrated with a case study free from mundane data manipulation chores.
Learning about XPath
Extract a single value (highest current minimum wage)
Lorem ipsum dolor sit amet, consectetur adipisicing elit. At, quasi accusantium quia numquam ex aliquid error quisquam recusandae doloribus amet animi quod autem assumenda cum. Aliquam eveniet, temporibus accusamus cupiditate.
Visualization of current states
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Dignissimos dolorum sed est fugiat ipsa minima nesciunt dolor veritatis beatae, labore a, laudantium nostrum dicta consequuntur quam aperiam hic at modi.
Consolidate multiple historical tables
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Eos id, beatae aliquid! Expedita obcaecati, sapiente alias, doloribus, modi illo enim, rerum officiis error reprehenderit perspiciatis impedit amet earum accusantium molestias.
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Et autem consectetur consequatur reprehenderit nemo iure inventore esse quasi, officiis cumque excepturi. Quae nesciunt, sunt, dolor unde harum fuga laboriosam sequi!
Better xpath example
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Culpa obcaecati quod blanditiis magni vero unde, accusamus impedit, perferendis maiores quisquam numquam. Minima adipisci, saepe sequi esse placeat quod nihil soluta.
Small multiples
Lorem ipsum dolor sit amet, consectetur adipisicing elit. Enim quas temporibus tempora eveniet excepturi. Numquam voluptate velit accusamus doloribus, quas, ea quisquam, totam commodi eveniet ipsum iure facere tempore consequatur.