Simple Simon met a pieman going to the fair;
Said Simple Simon to the pieman "Let me taste your ware"
Said the pieman to Simple Simon "Show me first your penny"
Said Simple Simon to the pieman "Sir, I have not any!"

and the following query q:

simple simple simon

We will calculate similarity between them using normalized similarity formula

The assignment requires that we use term frequencies as weights, i.e. q_i is term frequency for the i^th term in the query, and d_i is term frequency for the i^th term in the document. For the above example, the values are as follows.

Query:
qsimple = 2		qsimon = 1
All other qi values are 0.

Document:
dsimple = 4 dsimon = 4 dmet = 1 da = 1 dpieman = 4 dgoing = 1 dto = 4 dthe = 4 dfair = 1 dsaid = 3 dlet = 1 dme = 2		dtaste = 1 dyour = 2 dware = 1 dshow = 1 dfirst = 1 dpenny = 1 dsir = 1 dI = 1 dhave = 1 dnot = 1 dany = 1

Now when we know all the q_i and d_i values, we calculate the numerator (täljare) in the similarity formula:

q_simple · d_simple + q_simon · d_simon + q_met · d_met + q_a · d_a + q_pieman · d_pieman + q_going · d_going + q_to · d_to + q_the · d_the + q_fair · d_fair + q_said · d_said + q_let · d_let + q_me · d_me + q_taste · d_taste + q_your · d_your + q_ware · d_ware + q_show · d_show + q_first · d_first + q_penny · d_penny + q_sir · d_sir + q_I · d_I + q_have · d_have + q_not · d_not + q_any · d_any =
2 · 4 + 1 · 4 + 0 · 1 + 0 · 1 + 0 · 4 + 0 · 1 + 0 · 4 + 0 · 4 + 0 · 1 + 0 · 3 + 0 · 1 + 0 · 2 + 0 · 1 + 0 · 2 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 + 0 · 1 = 12

N_d in the similarity formula is the number of words in the document. The assignment says we count stop-words as well, therefore in the above example N_d = 42.

Finally, we calculate sim(q, d) as 12/42 = 0.28571.

---

Assignment 2