potential typo in 01- word2vec skipgram (code and lecture)

[Nevermetyou65]

Hi,

In the notebook of 01-word2vec notebook
there is a function

def random_batch(batch_size, word_sequence):
    
    # Make skip gram of one size window
    skip_grams = []
    # loop each word sequence
    # we starts from 1 because 0 has no context
    # we stop at second last for the same reason
    for sent in corpus:
        for i in range(1, len(sent) - 1):
            target = word2index[sent[i]]
            context = [word2index[sent[i - 1]], word2index[sent[i + 1]]]
            for w in context:
                skip_grams.append([target, w])
    
    random_inputs = []
    random_labels = []
    random_index = np.random.choice(range(len(skip_grams)), batch_size, replace=False) #randomly pick without replacement
        
    for i in random_index:
        random_inputs.append([skip_grams[i][0]])  # target, e.g., 2
        random_labels.append([skip_grams[i][1]])  # context word, e.g., 3
            
    return np.array(random_inputs), np.array(random_labels)

I think the definition of function should be

def random_batch(
    batch_size: int, 
    corpus: list[list[str]]
):

because the arg word_sequence isn't used anywhere in the function.

And in the lecture "L1 Word Vectors - Word2vec", page 14, objective function

For each position t = 1,...T, predict context words within a window of fixed size m, given center word wj , the loss function J(θ) is:

I think it should be "given center word wt" more than wj, if we reference content from the previous slide because when j=0, there is only center word.

I am sorry if I am mistaken
THANKS!!

[chaklam-silpasuwanchai]

Thank you so much!I will look into this and will let you know.Bests,Chaklam On 2 Jan BE 2568, at 09:52, pakpoom S. ***@***.***> wrote: Hi, In the notebook of 01-word2vec notebook there is a function def random_batch(batch_size, word_sequence): # Make skip gram of one size window skip_grams = [] # loop each word sequence # we starts from 1 because 0 has no context # we stop at second last for the same reason for sent in corpus: for i in range(1, len(sent) - 1): target = word2index[sent[i]] context = [word2index[sent[i - 1]], word2index[sent[i + 1]]] for w in context: skip_grams.append([target, w]) random_inputs = [] random_labels = [] random_index = np.random.choice(range(len(skip_grams)), batch_size, replace=False) #randomly pick without replacement for i in random_index: random_inputs.append([skip_grams[i][0]]) # target, e.g., 2 random_labels.append([skip_grams[i][1]]) # context word, e.g., 3 return np.array(random_inputs), np.array(random_labels) I think the definition of function should be def random_batch( batch_size: int, corpus: list[list[str]] ): because the arg word_sequence isn't used anywhere in the function. And in the lecture "L1 Word Vectors - Word2vec", page 14, objective function For each position t = 1,...T, predict context words within a window of fixed size m, given center word wj , the loss function J(θ) is: I think it should be "given center word wt" more than wj, if we reference content from the previous slide because when j=0, there is only center word. I am sorry if I am mistaken THANKS!! —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>