Aggregation functions in PIG

Aggregation functions in PIG

  4:12 PM    ssaikgame

---

Example1: year price 2015 60 2014 45 2016 34 2014 75 2015 45 2014 41 I would like to get the result as, total price of the item by year. grunt> file = load '/tmp/yearprice.txt' using PigStorage(',') as (year:int,price:float); grunt> data = foreach file generate year, price by $0 > 1; grunt> group = group data by year; grunt> sum = foreach group generate group...

Read More

PIG left, right outer joins error

PIG left, right outer joins error

  5:09 PM    ssaikgame

---

We know that, we can perform JOIN operation using pig, as well as left outer join, right outer join. sometimes you'll face error when working of left and right outer joins, but not to inner joins. that is because of schema. Even if there is no schema can perform inner...

Read More

What is HTTP Tunneling - Connections Through Restrictions

What is HTTP Tunneling - Connections Through Restrictions

  3:34 PM    ssaikgame

---

It is used to bypass firewalls and other network restrictions and an HTTP tunnel is used to create a direct network link between two locations. But before proceeding further, let us look into the meaning of some of the terms we will use in this context. Commonly...

Read More

Advanced scripts - PIG Part 3

Advanced scripts - PIG Part 3

  9:53 PM    ssaikgame

---

FILTER: filter = FILTER climate by country MATCHES 'C.*a'; -> china filter = FILTER climate by country MATCHES '.*(nda|hin).*; -> india, china SPLIT: SPLIT climate into B1992 if $1 == 1992, B2002 if $1 == 2002; it split the climate bag into two records B1992 and...

Read More

PIG Transformations or Operators - Part 2

PIG Transformations or Operators - Part 2

  2:51 AM    ssaikgame

---

LOAD: Loading the data into “base” grunt> base = load 'filepig.txt' using PigStorage('|') as (empid:int,ename:chararray,salary:int); PigStorage(‘|’) è is used to separate the columns using delimiter ‘|’ The filepig.txt contains 3columns separated with delimiter...

Read More