Documents:
Doc#1:
Cats are cool, soft, fuzzy and bouncy!
Doc#2:
Dogs love to eat and run around.
Doc#3:
It was raining cats and dogs the other night... so bad that I couldn't go outside. Sometimes I would come to the window and just stare at the rain. It was very depressing, but in the morning, I felt better!
Doc#4:
It's a dog-eat-dog world out there. From puppies to big hounds, everyone struggles to survive, to avoid his superior and to beat up on his inferior. That's just how it is.
Doc#5:
Cats and dogs are two common types of household animals. There are many species of cats and dogs - from the common house cat, to the Blue Russian, from bulldog to shepherd. Both cats and dogs have been domesticated by man many thousands of years ago and are loved and cared for by many pet owners today. There are even urban legends of cat owners having statistically better health than non-cat owners - and everyone knows how useful a dog can be, for protecting the house, for instance! There are many more things to say about cats and dogs, but I think I've run out of time, so I have to go. Thank you for listening!
Using
http://rainbow.arch.scriptmania.com/tools/word_counter.html Doc#1:
Unique:7 Total:7
Freq. Word
1 CATS
1 ARE
1 COOL
1 SOFT
1 FUZZY
1 AND
1 BOUNCY
Doc #2
Unique:7 Total:7
Freq. Word
1 DOGS
1 LOVE
1 TO
1 EAT
1 AND
1 RUN
1 AROUND
Doc #3
Unique words:33 Total words:41
Freq. Word
4 THE
3 I
2 AND
2 IT
2 WAS
1 DOGS
1 CATS
1 OTHER
1 NIGHT
1 SO
1 BAD
1 THAT
1 RAINING
1 COULDN
1 T
1 GO
1 OUTSIDE
1 SOMETIMES
1 WOULD
1 COME
1 TO
1 WINDOW
1 JUST
1 STARE
1 AT
1 RAIN
1 VERY
1 DEPRESSING
1 BUT
1 IN
1 MORNING
1 FELT
1 BETTER
Doc #4
Unique words:28 Total words:35
Freq. Word
4 TO
2 S
2 DOG
2 IT
2 HIS
1 WORLD
1 OUT
1 THERE
1 FROM
1 PUPPIES
1 A
1 BIG
1 HOUNDS
1 EVERYONE
1 STRUGGLES
1 SURVIVE
1 AVOID
1 EAT
1 SUPERIOR
1 AND
1 BEAT
1 UP
1 ON
1 INFERIOR
1 THAT
1 JUST
1 HOW
1 IS
Doc #5
Unique words:72 Total words:116
Freq. Word
7 AND
5 ARE
5 OF
4 CATS
4 DOGS
4 MANY
4 TO
4 FOR
3 CAT
3 THERE
3 THE
3 OWNERS
3 I
2 HAVE
2 BY
2 HOUSE
2 FROM
2 COMMON
1 BLUE
1 RUSSIAN
1 BULLDOG
1 SHEPHERD
1 BOTH
1 HOUSEHOLD
1 BEEN
1 DOMESTICATED
1 ANIMALS
1 MAN
1 THOUSANDS
1 YEARS
1 AGO
1 LOVED
1 CARED
1 TYPES
1 PET
1 TWO
1 TODAY
1 EVEN
1 URBAN
1 LEGENDS
1 HAVING
1 STATISTICALLY
1 BETTER
1 HEALTH
1 THAN
1 NON
1 EVERYONE
1 KNOWS
1 HOW
1 USEFUL
1 A
1 DOG
1 CAN
1 BE
1 PROTECTING
1 INSTANCE
1 MORE
1 THINGS
1 SAY
1 ABOUT
1 BUT
1 SPECIES
1 THINK
1 VE
1 RUN
1 OUT
1 TIME
1 SO
1 GO
1 THANK
1 YOU
1 LISTENING
All Docs:
Unique words:117 Total words:206
Freq. Word
12 AND
10 TO
7 THE
6 DOGS
6 ARE
6 CATS
6 I
5 OF
4 THERE
4 IT
4 MANY
4 FOR
3 FROM
3 CAT
3 DOG
3 OWNERS
2 BETTER
2 S
2 A
2 WAS
2 OUT
2 RUN
2 SO
2 EVERYONE
2 HIS
2 HOW
2 COMMON
2 THAT
2 EAT
2 HOUSE
2 GO
2 HAVE
2 BY
2 JUST
2 BUT
1 VERY
1 DEPRESSING
1 RAINING
1 IN
1 MORNING
1 FELT
1 FUZZY
1 OTHER
1 NIGHT
1 COOL
1 WORLD
1 BAD
1 BOUNCY
1 AROUND
1 PUPPIES
1 BIG
1 HOUNDS
1 COULDN
1 STRUGGLES
1 SURVIVE
1 AVOID
1 T
1 SUPERIOR
1 BEAT
1 UP
1 ON
1 INFERIOR
1 SOFT
1 IS
1 TWO
1 OUTSIDE
1 TYPES
1 SOMETIMES
1 HOUSEHOLD
1 ANIMALS
1 WOULD
1 SPECIES
1 COME
1 WINDOW
1 BLUE
1 RUSSIAN
1 BULLDOG
1 SHEPHERD
1 BOTH
1 LOVE
1 BEEN
1 DOMESTICATED
1 STARE
1 MAN
1 THOUSANDS
1 YEARS
1 AGO
1 LOVED
1 CARED
1 AT
1 PET
1 RAIN
1 TODAY
1 EVEN
1 URBAN
1 LEGENDS
1 HAVING
1 STATISTICALLY
1 HEALTH
1 THAN
1 NON
1 KNOWS
1 USEFUL
1 CAN
1 BE
1 PROTECTING
1 INSTANCE
1 MORE
1 THINGS
1 SAY
1 ABOUT
1 THINK
1 VE
1 TIME
1 THANK
1 YOU
1 LISTENING
Then using Porter Stemming online to stem and remove stop words:
http://maya.cs.depaul.edu/~classes/ds575/porter.html Doc#1:
| Original Word |
Stemmed Word |
| cats |
cat |
| cool |
cool |
| soft |
soft |
| fuzzy |
fuzzi |
| bouncy |
bounci |
Unique words:5 Total words:5
Freq. Word
1 CAT
1 COOL
1 SOFT
1 FUZZI
1 BOUNCI
Doc#2:
| Original Word |
Stemmed Word |
| dogs |
dog |
| love |
love |
| eat |
eat |
| run |
run |
Unique words:4 Total words:4
Freq. Word
1 DOG
1 LOVE
1 EAT
1 RUN
Doc#3:
| Original Word |
Stemmed Word |
| raining |
rain |
| cats |
cat |
| dogs |
dog |
| night |
night |
| bad |
bad |
| couldnt |
couldnt |
| go |
go |
| outside |
outsid |
| sometimes |
sometim |
| window |
window |
| stare |
stare |
| rain |
rain |
| depressing |
depress |
| morning |
morn |
| felt |
felt |
| better |
better |
Unique words:15 Total words:16
Freq. Word
2 RAIN
1 CAT
1 DOG
1 NIGHT
1 BAD
1 COULDNT
1 GO
1 OUTSID
1 SOMETIM
1 WINDOW
1 STARE
1 DEPRESS
1 MORN
1 FELT
1 BETTER
Doc#4:
| Original Word |
Stemmed Word |
| dogeatdog |
dogeatdog |
| world |
world |
| puppies |
puppi |
| big |
big |
| hounds |
hound |
| everyone |
everyon |
| struggles |
struggl |
| survive |
surviv |
| avoid |
avoid |
| superior |
superior |
| beat |
beat |
| inferior |
inferior |
| thats |
that |
Unique words:13 Total words:13
Freq. Word
1 DOGEATDOG
1 WORLD
1 PUPPI
1 BIG
1 HOUND
1 EVERYON
1 STRUGGL
1 SURVIV
1 AVOID
1 SUPERIOR
1 BEAT
1 INFERIOR
1 THAT
Doc#5:
| Original Word |
Stemmed Word |
| cats |
cat |
| dogs |
dog |
| common |
common |
| types |
type |
| household |
household |
| animals |
anim |
| many |
mani |
| species |
speci |
| cats |
cat |
| dogs |
dog |
| common |
common |
| house |
hous |
| cat |
cat |
| blue |
blue |
| russian |
russian |
| bulldog |
bulldog |
| shepherd |
shepherd |
| cats |
cat |
| dogs |
dog |
| domesticated |
domest |
| man |
man |
| many |
mani |
| thousands |
thousand |
| years |
year |
| ago |
ago |
| loved |
love |
| cared |
care |
| many |
mani |
| pet |
pet |
| owners |
owner |
| today |
todai |
| even |
even |
| urban |
urban |
| legends |
legend |
| cat |
cat |
| owners |
owner |
| statistically |
statist |
| better |
better |
| health |
health |
| noncat |
noncat |
| owners |
owner |
| everyone |
everyon |
| knows |
know |
| useful |
us |
| dog |
dog |
| protecting |
protect |
| house |
hous |
| instance |
instanc |
| many |
mani |
| more |
more |
| things |
thing |
| cats |
cat |
| dogs |
dog |
| think |
think |
| ive |
iv |
| run |
run |
| time |
time |
| go |
go |
| thank |
thank |
| listening |
listen |
Unique words:44 Total words:60
Freq. Word
6 CAT
5 DOG
4 MANI
3 OWNER
2 HOUS
2 COMMON
1 TYPE
1 SPECI
1 HOUSEHOLD
1 BLUE
1 RUSSIAN
1 BULLDOG
1 SHEPHERD
1 DOMEST
1 MAN
1 THOUSAND
1 YEAR
1 AGO
1 LOVE
1 CARE
1 PET
1 ANIM
1 TODAI
1 EVEN
1 URBAN
1 LEGEND
1 STATIST
1 BETTER
1 HEALTH
1 NONCAT
1 EVERYON
1 KNOW
1 US
1 PROTECT
1 INSTANC
1 MORE
1 THING
1 THINK
1 IV
1 RUN
1 TIME
1 GO
1 THANK
1 LISTEN
All Docs:
| Original Word |
Stemmed Word |
| cats |
cat |
| cool |
cool |
| soft |
soft |
| fuzzy |
fuzzi |
| bouncy |
bounci |
| dogs |
dog |
| love |
love |
| eat |
eat |
| run |
run |
| raining |
rain |
| cats |
cat |
| dogs |
dog |
| night |
night |
| bad |
bad |
| couldnt |
couldnt |
| go |
go |
| outside |
outsid |
| sometimes |
sometim |
| window |
window |
| stare |
stare |
| rain |
rain |
| depressing |
depress |
| morning |
morn |
| felt |
felt |
| better |
better |
| dogeatdog |
dogeatdog |
| world |
world |
| puppies |
puppi |
| big |
big |
| hounds |
hound |
| everyone |
everyon |
| struggles |
struggl |
| survive |
surviv |
| avoid |
avoid |
| superior |
superior |
| beat |
beat |
| inferior |
inferior |
| thats |
that |
| cats |
cat |
| dogs |
dog |
| common |
common |
| types |
type |
| household |
household |
| animals |
anim |
| many |
mani |
| species |
speci |
| cats |
cat |
| dogs |
dog |
| common |
common |
| house |
hous |
| cat |
cat |
| blue |
blue |
| russian |
russian |
| bulldog |
bulldog |
| shepherd |
shepherd |
| cats |
cat |
| dogs |
dog |
| domesticated |
domest |
| man |
man |
| many |
mani |
| thousands |
thousand |
| years |
year |
| ago |
ago |
| loved |
love |
| cared |
care |
| many |
mani |
| pet |
pet |
| owners |
owner |
| today |
todai |
| even |
even |
| urban |
urban |
| legends |
legend |
| cat |
cat |
| owners |
owner |
| statistically |
statist |
| better |
better |
| health |
health |
| noncat |
noncat |
| owners |
owner |
| everyone |
everyon |
| knows |
know |
| useful |
us |
| dog |
dog |
| protecting |
protect |
| house |
hous |
| instance |
instanc |
| many |
mani |
| more |
more |
| things |
thing |
| cats |
cat |
| dogs |
dog |
| think |
think |
| ive |
iv |
| run |
run |
| time |
time |
| go |
go |
| thank |
thank |
| listening |
listen |
Unique words:72 Total words:98
Freq. Word
8 CAT
7 DOG
4 MANI
3 OWNER
2 RAIN
2 GO
2 BETTER
2 EVERYON
2 COMMON
2 LOVE
2 HOUS
2 RUN
1 COULDNT
1 COOL
1 OUTSID
1 SOMETIM
1 WINDOW
1 STARE
1 DEPRESS
1 MORN
1 FELT
1 SOFT
1 DOGEATDOG
1 WORLD
1 PUPPI
1 BIG
1 HOUND
1 EAT
1 STRUGGL
1 SURVIV
1 AVOID
1 SUPERIOR
1 BEAT
1 INFERIOR
1 THAT
1 FUZZI
1 TYPE
1 HOUSEHOLD
1 ANIM
1 BOUNCI
1 SPECI
1 NIGHT
1 BLUE
1 RUSSIAN
1 BULLDOG
1 SHEPHERD
1 DOMEST
1 MAN
1 THOUSAND
1 YEAR
1 AGO
1 CARE
1 PET
1 BAD
1 TODAI
1 EVEN
1 URBAN
1 LEGEND
1 STATIST
1 HEALTH
1 NONCAT
1 KNOW
1 US
1 PROTECT
1 INSTANC
1 MORE
1 THING
1 THINK
1 IV
1 TIME
1 THANK
1 LISTEN