Tratamento de Dados com PIG, Zeppelin e Hive

Prévia do material em texto

UNIVERSIDADE ESTÁCIO DE SÁ
ESPECIALIZAÇÃO CIÊNCIA DE DADOS E BIG DATA ANALYTICS
Tratamento de dados HDFS 
HENRIQUE MATEUS FRANZE
Trabalho da disciplina NPG2062
 		 Tutor: Prof. DENIS GONCALVES COPLE
ARTHUR NOGUEIRA 
2019
15
Objetivo:
Usando 3 das tecnologias utilizadas durante a disciplina tratar os dados de uma base de dados.
Proposta:
Utilizando os dados de população por sexo, área (rural ou urbana) e idade e ano de United Nations Statistics Division (UNData), verificar se existe tendência de movimentação da maior parte da população entre área, idade e sexo.
Os dados:
Os dados podem ser baixados em:
http://data.un.org/Data.aspx?d=POP&f=tableCode%3a22%3bcountryCode%3a76%3bareaCode%3a1%2c2%2c3%3bsexCode%3a1%2c2&c=2,3,6,8,10,16&s=_countryEnglishNameOrderBy:asc,refYear:desc,areaCode:asc&v=1
Com o link acima, clicando em baixar se obterá os dados para o Brasil, durante esse trabalho foram usados os dados do Brasil e do mundo, bastando para isso retirar o Brasil dos filtros, aplicar a alteração do filtro e baixar novamente os dados em formato .csv:
Imagem 1 – Obtenção dos dados
Os dados se referem ao total da população (Value) para cada país, ano do censo, sexo e idade.
Para a idade temos:
· Os valores da idade explícitos 1, 2, 3...
· Faixas de idade 0 – 4, 5 – 9, 20 – 25 ...
· E o Total que excluiremos das nossas consultas durante o tratamento de dados.
Total de cerca de 99 mil dados com pouco mais de 5MB.
PIG Latin:
Foram usados os seguintes passos para tratamento dos dados usando o PIG:
1 - Registramos o seguinte requisito afim de auxiliar no carregamento do arquivo CSV com header:
REGISTER 'piggybank.jar';
2 – Leitura dos dados com a declaração das colunas e dos tipos:
A = LOAD 'UNdata_global.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage(',', 'NO_MULTILINE', 'UNIX', 'SKIP_INPUT_HEADER') AS (Country: chararray, Year: int, Area: chararray, Sex: chararray, Age: chararray, Value: int, foot: int);;
3 – Como citamos anteriormente não queremos enxergar as linhas de total de população para todas as idades:
B = FILTER A BY NOT Age MATCHES 'Total';
4 – Agrupamos os dados por país Country, Year
G2 = GROUP B by (Country, Year);
5 – Para cada Grupo G2, vamos ordenar por valor descendente e pegar somente a primeira linha, que é a linha da Area, Sexo e Idade mais populosa para aquele País e Ano:
result = FOREACH G2 {
 sortByMax = ORDER B BY Value DESC;
 topMax = LIMIT sortByMax 1;
 GENERATE FLATTEN(topMax.$0), FLATTEN(topMax.$1), FLATTEN(topMax.$2), FLATTEN(topMax.$3), FLATTEN(topMax.$4), FLATTEN(topMax.$5);
 }
6 – Mostramos na tela os resultados:
DUMP result;
Para executar o script acima foi gerado um arquivo UNdata.pig e executado com PIG –f UNdata.pig:
Imagem 2 – execução PIG Latin
Note os resultados (Maiores considerações sobre o resultado serão explanadas na conclusão):
Imagem 3 – resultado PIG latin
ZEPPELIN:
1 – Carregamento dos dados em texto, para visualização, (desconsidere a coluna Footnotes gerada automaticamente pelo site UNdata, não serão usados):
Imagem 4 – carregando csv Zeppelin
Obs: Foi necessário apagar as notas de rodapé e as linhas em branco do final do arquivo na mão, o que não é ideal pois geralmente dados são apresentados em arquivos muito maiores que não possibilita essa opção, mas alternativas devem ser consideradas ao utilizar em produção.
2 - Importação dos dados para tabela temporária:
Nessa fase foi feito:
a. Os dados foram separados por vírgula (,);
b. Foi filtrada a linha que inicia com “Country or Area” excluindo-se o header do CSV;
c. Foram filtrados os dados que iniciavam com (“China) com aspas somente no início pois havia uma seria de valores para País que usava virgula e estava dando erro;
d. Foi retirado as aspas de todos os valores com replaceAll;
e. Os dados do Ano foram convertidos em inteiro;
f. Os dados de valor foram convertidos para Float (poderia ser inteiro, mas tem alguns valores não inteiros);
Imagem 5 – Importação dos dados Zeppelin
Detalhe com o problema com o valor com virgula:
Imagem 6 – detalhe problema com dados separados por vírgula.
No CSV vem assim: “China, Hong Kong SAR”, esses valores foram filtrados.
3 – Consulta:
Foi criada a coluna row (linha) para os dados particionados por país e ano e ordenados por valor descendente. Desta forma pegando somente a primeira linha de cada partição temos o valor máximo para aquele país e ano e os dados correspondentes.
Note que os dados com a idade “Total” foram excluídos dessa consulta pois o Total é sempre o registro maior e ele não nos interessa nesse contexto.
A análise dos dados do resultado será abordada na conclusão.
Imagem 7 – Consulta Zeppelin
Hive:
Os dados foram colocados no HDFS usando a interface gráfica na pasta /user/root/UNfolder
Imagem 8 – Dados HDFS para o Hive
Criação da tabela a partir dos dados da pasta com o LOCATION:
Imagem 9 – Criação tabela temporária a partir dos dados Hive
Consulta:
Imagem 10 – consulta executada Hive
Resultado:
Imagem 11 – Resultado dos dados no Hive
CONCLUSÃO:
Comparando-se as imagens 4, 7 e 11 observamos que chegamos para o mesmo resultado utilizando os 3 métodos, veja em detalhe os resultados para o Brasil:
	country
	year
	area
	sex
	age
	value
	Brazil
	1960
	Rural
	Male
	0 - 4
	3341939
	Brazil
	1970
	Urban
	Male
	43713
	3501168
	Brazil
	1980
	Urban
	Male
	0 - 4
	5187257
	Brazil
	1991
	Urban
	Male
	43713
	6301508
	Brazil
	1996
	Urban
	Male
	41913
	6615804
	Brazil
	2000
	Urban
	Female
	15 - 19
	7270717
	Brazil
	2010
	Urban
	Female
	25 - 29
	7547225
Tabela 1 – Resultado para o Brasil
Note que a análise demonstra que desde 1960 até 2010 houve uma movimentação da parcela mais numerosa da população:
Zona Rural -> Zona Urbana
Crianças -> Adultos entre 25 – 29 anos
Homens -> Mulheres
Essa análise sozinha não demonstra mais intimamente a relação entre as variáveis, mas baseado no que conhecemos sobre a história podemos constatar que corresponde à devida a urbanização e melhorias do sistema de saúde.
A mesma tendência pode ser observada em outros países mais ou menos claramente conforme os dados informados e as particularidades do país.
Como curiosidade segue abaixo resultado completo.
RESULTADO COMPLETO:
country,year,area,sex,age,row,value
Afghanistan,1979,Rural,Female,1 - 4,1,892742.0
Albania,1955,Rural,Male,5 - 9,1,66522.0
Albania,2001,Rural,Male,10 - 14,1,102140.0
Albania,2002,Rural,Male,10 - 14,1,99460.21
Albania,2003,Rural,Male,10 - 14,1,95788.98
Albania,2004,Rural,Male,15 - 19,1,91807.9
Albania,2005,Rural,Male,15 - 19,1,90978.445
Albania,2006,Rural,Male,15 - 19,1,89655.45
Albania,2007,Rural,Male,15 - 19,1,87404.94
Albania,2008,Rural,Male,15 - 19,1,83501.17
Albania,2009,Rural,Male,15 - 19,1,79076.55
Albania,2010,Rural,Male,15 - 19,1,75099.625
Albania,2011,Rural,Male,15 - 19,1,71009.56
Albania,2012,Urban,Female,15 - 19,1,68627.445
Albania,2013,Urban,Female,20 - 24,1,78280.13
Algeria,1966,Rural,Male,1 - 4,1,575141.0
Algeria,2008,Urban,Female,20 - 24,1,1189553.2
American Samoa,1990,Rural,Male,1 - 4,1,2022.0
Andorra,1987,Urban,Male,25 - 29,1,1889.0
Andorra,1988,Urban,Male,30 - 34,1,1915.0
Andorra,1989,Urban,Male,30 - 34,1,2705.0
Andorra,1990,Urban,Male,30 - 34,1,2788.0
Andorra,1991,Urban,Male,25 - 29,1,3385.0
Angola,2014,Urban,Female,0 - 4,1,1461924.0
Antigua and Barbuda,1960,Rural,Male,5 - 9,1,2550.0
Antigua and Barbuda,1970,Rural,Male,5 - 9,1,3350.0
Argentina,1980,Urban,Male,0 - 4,1,1278679.0
Argentina,1985,Urban,Male,0 - 4,1,1443366.0
Argentina,1988,Urban,Male,0 - 4,1,1342723.0
Argentina,1991,Urban,Male,10 - 14,1,1437230.0
Argentina,1995,Urban,Male,15 - 19,1,1473952.0
Argentina,2005,Urban,Male,10 - 14,1,1556344.0
Argentina,2007,Urban,Male,10 - 14,1,1561531.0
Argentina,2009,Urban,Male,15 - 19,1,1578132.0
Argentina,2010,Urban,Male,15 - 19,1,1598214.0
Argentina,2014,Urban,Male,0 - 4,1,1727615.0
Argentina,2015,Urban,Male,0 - 4,1,1738699.0
Argentina,2016,Urban,Male,0 - 4,1,1734540.0
Argentina,2017,Urban,Male,0 - 4,1,1731865.0Argentina,2018,Urban,Male,0 - 4,1,1728491.0
Armenia,1959,Rural,Male,0 - 4,1,84122.0
Armenia,1970,Urban,Male,5 - 9,1,93838.0
Armenia,1979,Urban,Female,20 - 24,1,118962.0
Armenia,1989,Urban,Male,5 - 9,1,115216.0
Armenia,1991,Urban,Female,30 - 34,1,131800.0
Armenia,1992,Urban,Female,30 - 34,1,131574.0
Armenia,1996,Urban,Male,10 - 14,1,128923.0
Armenia,1997,Urban,Male,10 - 14,1,128498.0
Armenia,1998,Urban,Male,10 - 14,1,128085.0
Armenia,1999,Urban,Male,10 - 14,1,126217.0
Armenia,2000,Urban,Male,15 - 19,1,126344.0
Armenia,2001,Urban,Female,15 - 19,1,101350.0
Armenia,2004,Urban,Male,15 - 19,1,98248.0
Armenia,2006,Urban,Female,20 - 24,1,99688.0
Armenia,2007,Urban,Female,20 - 24,1,99501.0
Armenia,2008,Urban,Male,20 - 24,1,98124.0
Armenia,2009,Urban,Male,20 - 24,1,97520.0
Armenia,2011,Urban,Female,20 - 24,1,92031.0
Armenia,2015,Urban,Female,25 - 29,1,90350.0
Armenia,2016,Urban,Female,25 - 29,1,89185.0
Armenia,2017,Urban,Female,30 - 34,1,89801.0
Australia,1954,Urban,Male,5 - 9,1,349398.0
Australia,1966,Urban,Male,5 - 9,1,477421.0
Australia,1971,Urban,Male,10 - 14,1,526086.0
Australia,1976,Urban,Male,10 - 14,1,538950.0
Australia,1981,Urban,Male,10 - 14,1,551055.0
Australia,1986,Urban,Male,15 - 19,1,569138.0
Australia,2001,Urban,Female,30 - 34,1,637633.0
Australia,2004,Urban,Female,30 - 34,1,682577.0
Australia,2006,Urban,Female,35 - 39,1,664951.0
Australia,2008,Urban,Male,20 - 24,1,692818.0
Australia,2009,Urban,Male,20 - 24,1,721335.0
Australia,2011,Urban,Male,25 - 29,1,754831.0
Australia,2012,Urban,Male,25 - 29,1,770404.0
Australia,2013,Urban,Male,25 - 29,1,782488.0
Australia,2014,Urban,Male,25 - 29,1,792934.0
Australia,2015,Urban,Male,25 - 29,1,795769.0
Australia,2016,Urban,Female,25 - 29,1,823507.0
Australia,2017,Urban,Female,25 - 29,1,841491.0
Austria,1951,Urban,Female,30 - 49,1,571027.0
Austria,1961,Rural,Male,0 - 9,1,348266.0
Austria,1971,Rural,Male,5 - 9,1,188403.0
Austria,1981,Rural,Male,15 - 19,1,172633.0
Austria,1991,Urban,Male,25 - 29,1,231995.0
Austria,2001,Urban,Male,35 - 39,1,239432.0
Austria,2011,Urban,Female,45 - 49,1,235992.0
Azerbaijan,1959,Rural,Male,0 - 4,1,196039.0
Azerbaijan,1970,Rural,Male,5 - 9,1,242312.0
Azerbaijan,1979,Rural,Male,10 - 14,1,226305.0
Azerbaijan,1989,Urban,Male,5 - 9,1,203382.0
Azerbaijan,1996,Rural,Male,5 - 9,1,222000.0
Azerbaijan,1997,Rural,Male,5 - 9,1,225400.0
Azerbaijan,1998,Rural,Male,5 - 9,1,238000.0
Azerbaijan,1999,Rural,Male,5 - 9,1,244500.0
Azerbaijan,2000,Urban,Male,10 - 14,1,239400.0
Azerbaijan,2001,Urban,Male,10 - 14,1,239200.0
Azerbaijan,2002,Rural,Male,10 - 14,1,240700.0
Azerbaijan,2003,Rural,Male,10 - 14,1,237400.0
Azerbaijan,2004,Urban,Male,15 - 19,1,240200.0
Azerbaijan,2007,Urban,Male,15 - 19,1,242000.0
Azerbaijan,2008,Urban,Male,15 - 19,1,238600.0
Azerbaijan,2009,Urban,Female,20 - 24,1,256533.0
Azerbaijan,2010,Urban,Female,20 - 24,1,256300.0
Azerbaijan,2012,Urban,Female,25 - 29,1,248873.0
Azerbaijan,2013,Urban,Female,25 - 29,1,255038.0
Azerbaijan,2015,Urban,Female,25 - 29,1,257523.0
Azerbaijan,2016,Urban,Female,25 - 29,1,253499.0
Azerbaijan,2017,Urban,Female,30 - 34,1,249400.0
Bahrain,1965,Urban,Male,21 - 30,1,16980.0
Bahrain,1971,Urban,Male,5 - 9,1,13142.0
Bangladesh,1974,Rural,Male,5 - 9,1,6112450.0
Bangladesh,1981,Rural,Male,0 - 4,1,6493986.0
Bangladesh,2001,Rural,Male,5 - 9,1,7062729.0
Bangladesh,2011,Rural,Male,5 - 9,1,7511728.0
Belarus,1959,Rural,Male,1 - 9,1,571972.0
Belarus,1970,Rural,Female,60 - 69,1,314070.0
Belarus,1979,Urban,Female,20 - 24,1,303716.0
Belarus,1987,Urban,Female,25 - 29,1,344378.0
Belarus,1989,Urban,Female,30 - 34,1,330231.0
Belarus,1991,Urban,Female,30 - 34,1,342275.0
Belarus,1992,Urban,Female,30 - 34,1,343927.0
Belarus,1993,Urban,Female,30 - 34,1,341510.0
Belarus,1996,Urban,Female,35 - 39,1,347227.0
Belarus,1997,Urban,Female,35 - 39,1,349394.0
Belarus,1998,Urban,Female,35 - 39,1,348266.0
Belarus,1999,Urban,Female,35 - 39,1,342836.0
Belarus,2000,Urban,Female,40 - 44,1,331033.0
Belarus,2002,Urban,Female,40 - 44,1,339283.0
Belarus,2003,Urban,Female,40 - 44,1,334645.0
Belarus,2004,Urban,Male,20 - 24,1,326350.0
Belarus,2006,Urban,Female,45 - 49,1,334665.0
Belarus,2007,Urban,Female,45 - 49,1,335230.0
Belarus,2008,Urban,Female,45 - 49,1,331092.0
Belarus,2009,Urban,Male,25 - 29,1,328473.0
Belarus,2010,Urban,Female,50 - 54,1,318664.0
Belarus,2011,Urban,Female,50 - 54,1,323086.0
Belarus,2012,Urban,Female,50 - 54,1,324344.0
Belarus,2014,Urban,Male,25 - 29,1,325264.0
Belarus,2015,Urban,Male,25 - 29,1,323775.0
Belarus,2017,Urban,Female,30 - 34,1,327799.0
Belarus,2018,Urban,Female,30 - 34,1,329518.0
Belgium,2001,Urban,Male,35 - 39,1,404960.0
Belgium,2007,Urban,Male,40 - 44,1,406505.0
Belgium,2009,Urban,Male,45 - 49,1,406393.0
Belgium,2011,Urban,Male,45 - 49,1,413196.0
Belize,1960,Urban,Female,5 - 9,1,3561.0
Benin,1979,Rural,Male,5 - 9,1,196674.0
Benin,1992,Rural,Male,5 - 9,1,325715.0
Benin,2002,Rural,Male,5 - 9,1,401231.0
Bermuda,1950,Rural,Male,0 - 4,1,2246.0
Bermuda,2010,Urban,Female,45 - 49,1,2920.0
Bermuda,2016,Urban,Female,55 - 59,1,2846.0
Bhutan,2005,Rural,Male,10 - 14,1,27824.0
Bhutan,2017,Rural,Male,25 - 29,1,24104.0
Bolivia (Plurinational State of),1976,Rural,Male,5 - 9,1,195491.0
Bolivia (Plurinational State of),1992,Urban,Male,5 - 9,1,239975.0
Bolivia (Plurinational State of),2001,Urban,Male,0 - 4,1,325641.0
Bolivia (Plurinational State of),2012,Urban,Female,15 - 19,1,385535.0
Botswana,1964,Rural,Female,5 - 9,1,43757.0
Botswana,1971,Rural,Female,0 - 4,1,45454.0
Botswana,1981,Rural,Male,5 - 9,1,66938.0
Botswana,1991,Rural,Male,5 - 9,1,62477.0
Botswana,2001,Urban,Female,15 - 19,1,60553.0
Botswana,2006,Urban,Female,20 - 24,1,67398.0
Botswana,2011,Urban,Female,25 - 29,1,78122.0
Brazil,1960,Rural,Male,0 - 4,1,3341939.0
Brazil,1970,Urban,Male,5 - 9,1,3501168.0
Brazil,1980,Urban,Male,0 - 4,1,5187257.0
Brazil,1991,Urban,Male,5 - 9,1,6301508.0
Brazil,1996,Urban,Male,10 - 14,1,6615804.0
Brazil,2000,Urban,Female,15 - 19,1,7270717.0
Brazil,2010,Urban,Female,25 - 29,1,7547225.0
Brunei Darussalam,1960,Rural,Male,5 - 9,1,4173.0
Brunei Darussalam,1971,Urban,Male,5 - 9,1,5992.0
Brunei Darussalam,1981,Urban,Male,20 - 24,1,7765.0
Brunei Darussalam,1991,Urban,Male,30 - 34,1,9933.0
Brunei Darussalam,2001,Urban,Male,0 - 14,1,37396.0
Brunei Darussalam,2011,Urban,Male,25 - 29,1,15337.0
Bulgaria,1956,Rural,Male,5 - 9,1,244250.0
Bulgaria,1965,Urban,Female,15 - 19,1,218458.0
Bulgaria,1966,Urban,Male,15 - 19,1,227249.0
Bulgaria,1967,Urban,Male,15 - 19,1,241774.0
Bulgaria,1968,Urban,Male,15 - 19,1,248415.0
Bulgaria,1969,Urban,Male,15 - 19,1,247638.0
Bulgaria,1970,Urban,Male,20 - 24,1,247248.0
Bulgaria,1971,Urban,Male,20 - 24,1,268444.0
Bulgaria,1972,Urban,Male,20 - 24,1,280270.0
Bulgaria,1973,Urban,Male,20 - 24,1,283392.0
Bulgaria,1974,Urban,Male,20 - 24,1,281678.0
Bulgaria,1975,Urban,Male,20 - 24,1,280591.0
Bulgaria,1976,Urban,Male,15 - 19,1,236084.0
Bulgaria,1977,Urban,Male,15 - 19,1,245125.0
Bulgaria,1978,Urban,Male,15 - 19,1,247310.0
Bulgaria,1979,Urban,Male,15 - 19,1,244630.0
Bulgaria,1980,Urban,Male,15 - 19,1,242937.0
Bulgaria,1981,Urban,Male,20 - 24,1,250700.0
Bulgaria,1982,Urban,Male,20 - 24,1,260723.0
Bulgaria,1983,Urban,Male,20 - 24,1,263863.0
Bulgaria,1984,Urban,Male,20 - 24,1,264513.0
Bulgaria,1985,Urban,Female,35 - 39,1,245055.0
Bulgaria,1986,Urban,Female,35 - 39,1,256143.0
Bulgaria,1987,Urban,Female,35 - 39,1,251996.0
Bulgaria,1988,Urban,Female,35 - 39,1,252445.0
Bulgaria,1989,Urban,Female,35 - 39,1,250672.0
Bulgaria,1990,Urban,Female,40 - 44,1,248566.0
Bulgaria,1992,Urban,Female,40 - 44,1,240614.0
Bulgaria,1993,Urban,Male,15 - 19,1,247159.0
Bulgaria,1994,Urban,Male,15 - 19,1,242629.0
Bulgaria,1995,Urban,Female,45 - 49,1,236802.0
Bulgaria,1996,Urban,Female,45 - 49,1,234676.0
Bulgaria,1997,Urban,Male,20 - 24,1,238338.0
Bulgaria,2000,Urban,Male,20 - 24,1,238995.0
Bulgaria,2001,Urban,Female,70 +,1,278903.0
Bulgaria,2002,Urban,Male,25 - 29,1,224440.0
Bulgaria,2003,Urban,Male,25 - 29,1,228183.0
Bulgaria,2004,Urban,Male,25 - 29,1,227918.5
Bulgaria,2005,Urban,Male,25 - 29,1,225635.0
Bulgaria,2006,Urban,Male,25 - 29,1,222903.0
Bulgaria,2011,Urban,Male,35- 39,1,218080.0
Bulgaria,2012,Urban,Male,35 - 39,1,221237.0
Bulgaria,2015,Urban,Male,35 - 39,1,216679.0
Bulgaria,2016,Urban,Male,40 - 44,1,212050.5
Bulgaria,2017,Urban,Male,40 - 44,1,215292.5
Bulgaria,2018,Urban,Male,40 - 44,1,216795.0
Burkina Faso,1975,Rural,Male,5 - 9,1,442315.0
Burkina Faso,1985,Rural,Male,5 - 9,1,656045.0
Burkina Faso,1996,Rural,Male,5 - 9,1,768749.0
Burkina Faso,2006,Rural,Male,0 - 4,1,1016801.0
Burundi,1965,Rural,Female,0 - 4,1,309200.0
Burundi,2008,Rural,Female,0 - 4,1,661814.0
Cabo Verde,1990,Rural,Male,5 - 9,1,14605.0
Cabo Verde,2000,Rural,Female,5 - 9,1,16459.0
Cabo Verde,2010,Urban,Male,20 - 24,1,17504.0
Cambodia,1962,Rural,Male,5 - 9,1,401790.0
Cambodia,1998,Rural,Male,5 - 9,1,765580.0
Cambodia,2008,Rural,Male,10 - 14,1,732337.0
Cameroon,1976,Rural,Male,5 - 9,1,403189.0
Cameroon,2005,Rural,Male,0 - 4,1,850403.0
Canada,1961,Urban,Male,0 - 4,1,779610.0
Canada,1966,Urban,Male,5 - 9,1,824175.0
Canada,1971,Urban,Male,10 - 14,1,849210.0
Canada,1976,Urban,Male,15 - 19,1,871380.0
Canada,1981,Urban,Female,20 - 24,1,946290.0
Canada,1986,Urban,Female,25 - 29,1,939320.0
Canada,1991,Urban,Female,30 - 34,1,977965.0
Canada,1992,Urban,Male,30,1,192931.0
Canada,2001,Urban,Female,40 - 44,1,1043510.0
Canada,2006,Urban,Female,40 - 44,1,1064540.0
Canada,2011,Urban,Female,45 - 49,1,1091500.0
Canada,2016,Urban,Female,50 - 54,1,1082250.0
Cayman Islands,2010,Urban,Female,35 - 39,1,3191.0
Cayman Islands,2016,Urban,Female,40 - 44,1,3813.106
Cayman Islands,2017,Urban,Male,45 - 49,1,3459.0
Central African Republic,1975,Rural,Male,5 - 9,1,97337.0
Central African Republic,1988,Rural,Male,5 - 9,1,116343.0
Chad,2009,Rural,Male,0 - 4,1,912299.3
Chile,1952,Urban,Female,0 - 9,1,431844.0
Chile,1960,Urban,Female,5 - 9,1,315497.0
Chile,1970,Urban,Female,5 - 9,1,450283.0
Chile,1976,Urban,Male,0 - 4,1,492126.0
Chile,1977,Urban,Female,10 - 14,1,470640.0
Chile,1978,Urban,Female,15 - 19,1,481458.0
Chile,1979,Urban,Female,15 - 19,1,501514.0
Chile,1980,Urban,Female,15 - 19,1,508054.0
Chile,1981,Urban,Female,15 - 19,1,510591.0
Chile,1982,Urban,Female,15 - 19,1,550775.0
Chile,1983,Urban,Female,20 - 24,1,510734.0
Chile,1984,Urban,Female,20 - 24,1,519965.0
Chile,1985,Urban,Male,0 - 4,1,564485.0
Chile,1988,Urban,Male,0 - 4,1,599034.0
Chile,1989,Urban,Male,0 - 4,1,608297.0
Chile,1990,Urban,Male,0 - 4,1,616094.0
Chile,1991,Urban,Male,0 - 4,1,624462.0
Chile,1992,Urban,Male,0 - 4,1,630651.0
Chile,1993,Urban,Male,0 - 4,1,620316.0
Chile,1995,Urban,Male,0 - 4,1,627179.0
Chile,1996,Urban,Male,0 - 4,1,626226.0
Chile,1997,Urban,Male,0 - 4,1,625356.0
Chile,1998,Urban,Male,5 - 9,1,624804.0
Chile,2000,Urban,Male,5 - 9,1,633532.0
Chile,2001,Urban,Male,5 - 9,1,632513.0
Chile,2002,Urban,Male,5 - 9,1,631543.0
Chile,2003,Urban,Male,10 - 14,1,646613.0
Chile,2005,Urban,Male,10 - 14,1,651748.0
Chile,2006,Urban,Male,15 - 19,1,640594.0
Chile,2007,Urban,Male,15 - 19,1,643199.0
Chile,2008,Urban,Male,15 - 19,1,645804.0
Chile,2009,Urban,Male,15 - 19,1,648409.0
Chile,2010,Urban,Male,15 - 19,1,651013.0
Chile,2011,Urban,Male,20 - 24,1,638930.0
Chile,2012,Urban,Male,20 - 24,1,641496.0
Chile,2013,Urban,Male,20 - 24,1,644063.0
Chile,2015,Urban,Male,25 - 29,1,658103.0
Chile,2016,Urban,Male,25 - 29,1,664541.0
Chile,2017,Urban,Male,25 - 29,1,666743.0
Chile,2018,Urban,Male,25 - 29,1,664985.0
China,1982,Rural,Male,10 - 14,1,5.6342236E7
China,1987,Rural,Male,15 - 19,1,4.2852E7
China,1990,Rural,Male,15 - 19,1,4.6794116E7
China,2000,Rural,Male,10 - 14,1,4.6643432E7
China,2010,Urban,Male,20 - 24,1,3.6041784E7
Colombia,1964,Rural,Male,5 - 9,1,731243.0
Colombia,1973,Urban,Female,10 - 14,1,927987.0
Colombia,1985,Urban,Female,15 - 19,1,1202412.0
Colombia,1993,Urban,Female,10 - 14,1,1308576.0
Colombia,2005,Urban,Male,10 - 14,1,1605865.0
Comoros,1980,Rural,Male,5 - 9,1,22832.0
Cook Islands,1981,Urban,Male,10 - 14,1,744.0
Costa Rica,1973,Rural,Male,5 - 9,1,97120.0
Costa Rica,1981,Rural,Male,0 - 11,1,200459.0
Costa Rica,1982,Rural,Male,0 - 12,1,200566.0
Costa Rica,1983,Rural,Male,0 - 14,1,237960.0
Costa Rica,1984,Rural,Male,0,1,20178.0
Costa Rica,1985,Rural,Male,5 - 9,1,90426.0
Costa Rica,1993,Rural,Female,30 - 39,1,118820.0
Costa Rica,1994,Rural,Female,30 - 39,1,126424.0
Costa Rica,1995,Rural,Female,30 - 39,1,133821.0
Costa Rica,1996,Rural,Female,30 - 39,1,137562.0
Costa Rica,1997,Rural,Female,30 - 39,1,142719.0
Costa Rica,1998,Rural,Female,30 - 39,1,149915.0
Costa Rica,2000,Urban,Male,10 - 14,1,121244.0
Costa Rica,2003,Urban,Male,15 - 19,1,128173.0
Costa Rica,2006,Urban,Female,40 - 49,1,188123.0
Costa Rica,2007,Urban,Female,40 - 49,1,187941.0
Costa Rica,2011,Urban,Female,20 - 24,1,153912.0
Costa Rica,2012,Urban,Male,20 - 24,1,142639.0
Costa Rica,2013,Urban,Female,20 - 24,1,164603.0
Costa Rica,2014,Urban,Male,20 - 24,1,162730.0
Costa Rica,2015,Urban,Female,20 - 24,1,160936.0
Costa Rica,2016,Urban,Female,20 - 24,1,161082.0
Costa Rica,2017,Urban,Male,15 - 19,1,155907.0
Costa Rica,2018,Urban,Male,20 - 24,1,155965.0
Croatia,1991,Urban,Female,35 - 39,1,113616.0
Croatia,2001,Urban,Female,45 - 49,1,101343.0
Croatia,2011,Urban,Female,50 - 54,1,93018.0
Cuba,1965,Urban,Female,15 - 59,1,1229610.0
Cuba,1966,Urban,Female,15 - 59,1,1255750.0
Cuba,1970,Urban,Male,5 - 9,1,341664.0
Cuba,1981,Urban,Male,15 - 19,1,398508.0
Cuba,1982,Urban,Male,15 - 19,1,412404.0
Cuba,1983,Urban,Male,15 - 19,1,419126.0
Cuba,1984,Urban,Male,15 - 19,1,408885.0
Cuba,1985,Urban,Male,15 - 19,1,402929.0
Cuba,1986,Urban,Female,20 - 24,1,423237.0
Cuba,1987,Urban,Female,20 - 24,1,422553.0
Cuba,1988,Urban,Male,20 - 24,1,420052.0
Cuba,1989,Urban,Male,20 - 24,1,418160.0
Cuba,1990,Urban,Female,25 - 29,1,413122.0
Cuba,1991,Urban,Female,25 - 29,1,432572.0
Cuba,1992,Urban,Female,25 - 29,1,441775.0
Cuba,1993,Urban,Female,25 - 29,1,437601.0
Cuba,1995,Urban,Female,25 - 29,1,421017.0
Cuba,1996,Urban,Female,30 - 34,1,433796.0
Cuba,1997,Urban,Female,30 - 34,1,443449.0
Cuba,1998,Urban,Female,30 - 34,1,438105.0
Cuba,1999,Urban,Female,30 - 34,1,428216.0
Cuba,2000,Urban,Female,30 - 34,1,417502.0
Cuba,2002,Urban,Female,35 - 39,1,441390.0
Cuba,2003,Urban,Female,35 - 39,1,437999.0
Cuba,2004,Urban,Female,35 - 39,1,424427.0
Cuba,2005,Urban,Female,35 - 39,1,409557.0
Cuba,2006,Urban,Female,40 - 44,1,422190.0
Cuba,2007,Urban,Female,40 - 44,1,431716.0
Cuba,2008,Urban,Female,40 - 44,1,427286.0
Cuba,2009,Urban,Female,40 - 44,1,414249.0
Cuba,2010,Urban,Female,40 - 44,1,400093.0
Cuba,2011,Urban,Female,45 - 49,1,410866.0
Cuba,2012,Urban,Female,45 - 49,1,423810.0
Cuba,2013,Urban,Female,45 - 49,1,421191.5
Cuba,2014,Urban,Female,45 - 49,1,412659.0
Cuba,2015,Urban,Female,45 - 49,1,403182.0
Cuba,2016,Urban,Female,50 - 54,1,407682.0
Cuba,2017,Urban,Female,50 - 54,1,417400.5
Cuba,2018,Urban,Female,50 - 54,1,415097.5
Cyprus,1960,Rural,Male,5 - 9,1,23827.0
Cyprus,1982,Urban,Female,20 - 24,1,15805.0
Cyprus,1992,Urban,Male,5 - 9,1,17837.0
Cyprus,2001,Urban,Female,35 - 39,1,19341.0
Cyprus,2011,Urban,Female,25 - 29,1,26013.0
Czechia,1994,Urban,Male,15 - 19,1,336662.0
Czechia,1995,Urban,Male,20 - 24,1,325447.0
Czechia,1996,Urban,Male,20 - 24,1,336913.0
Czechia,1997,Urban,Male,20 - 24,1,343499.0
Czechia,1998,Urban,Male,20 - 24,1,342255.0
Czechia,1999,Urban,Male,20 - 24,1,334536.0
Czechia,2000,Urban,Male,25 - 29,1,326976.0
Czechia,2001,Urban,Male,25 - 29,1,324524.0
Czechia,2011,Urban,Male,30 - 34,1,329673.0
Czechia,2012,Urban,Male,35 - 39,1,340997.0
Czechia,2013,Urban,Male,35 - 39,1,348236.0
Czechia,2014,Urban,Male,35 - 39,1,346655.0
Czechia,2015,Urban,Male,35 - 39,1,341222.0
Czechia,2016,Urban,Male,40 - 44,1,330098.5
Czechia,2017,Urban,Male,40 - 44,1,340241.5
Czechia,2018,Urban,Male,40 - 44,1,344129.0
Côte d'Ivoire,1975,Rural,Male,0 - 4,1,437779.0
Côte d'Ivoire,1978,Rural,Male,5 - 9,1,413225.0
Côte d'Ivoire,1988,Rural,Male,5 - 9,1,567127.0
Côte d'Ivoire,2014,Rural,Male,0 - 4,1,1079859.0
Democratic People's Republic of Korea,1993,Urban,Female,20 - 24,1,665054.0
Denmark,1960,Urban,Male,10 - 14,1,146993.0
Denmark,1965,Urban,Female,40 - 59,1,427885.0
Denmark,1968,Rural,Male,5 - 9,1,119693.0
Denmark,1969,Rural,Male,5 - 9,1,122854.0
Denmark,1970,Urban,Female,20- 24,1,154946.0
Denmark,1971,Rural,Male,5,1,39303.0
Denmark,1972,Rural,Male,6,1,39608.0
Denmark,1976,Urban,Male,30 - 34,1,177543.0
Dominican Republic,1950,Rural,Male,5 - 9,1,122883.0
Dominican Republic,1960,Rural,Male,5 - 9,1,180960.0
Dominican Republic,1970,Rural,Male,5 - 9,1,210521.0
Dominican Republic,1993,Urban,Female,20 - 24,1,246195.0
Dominican Republic,2002,Urban,Male,5 - 9,1,297879.0
Dominican Republic,2010,Urban,Female,15 - 19,1,370836.0
Ecuador,1982,Rural,Male,5 - 9,1,319449.0
Ecuador,1990,Urban,Male,5 - 9,1,321778.0
Ecuador,2001,Urban,Male,5 - 9,1,389609.0
Ecuador,2010,Urban,Male,10 - 14,1,463344.0
Åland Islands,2000,Rural,Male,50 - 54,1,628.0
Åland Islands,2007,Rural,Male,40 - 44,1,648.0
Åland Islands,2009,Rural,Female,40 - 44,1,658.0
Åland Islands,2010,Rural,Male,60 - 64,1,647.0
Åland Islands,2011,Rural,Male,45 - 49,1,661.5
Åland Islands,2012,Rural,Male,45 - 49,1,680.0
Åland Islands,2013,Rural,Male,45 - 49,1,690.0
Åland Islands,2014,Rural,Male,45 - 49,1,681.0
Åland Islands,2015,Rural,Male,45 - 49,1,657.0
Åland Islands,2016,Rural,Male,50 - 54,1,675.0
Åland Islands,2017,Rural,Male,50 - 54,1,685.0
Åland Islands,2018,Rural,Male,50 - 54,1,699.5
Bibliografia:
https://blogs.msdn.microsoft.com/avkashchauhan/2012/04/12/processing-million-songs-dataset-with-pig-scripts-on-apache-hadoop-on-windows-azure/
Apostila da Disciplina.