Managing computationally heavy analyses/functions

Set everything up by only using around 20 rows of your data

Manage time; the most computationally heavy and time consuming functions are textHuggingFace (which is also used in textEmbed). Hence, plan your analyses, and start them over a coffe break or over night.

To better be able to plan your analyses, take time to get a better idea of how long time different analyses will takes.

T1 <- Sys.time()
# Aggregating layer 11 and 12 by taking the mean of each dimension. 
we_11_12_mean <- textLayerAggregation(word_embeddings_layers = wordembeddings_tokens_layers,
                                      layers = 11:12,
                                      aggregation = "mean")
T2 <- Sys.time()

Your system’s capacity

Thinking about your computer’s memory capacity may be important if you have a lot of data and interested in many layers. For example, one sentence of 10 words/tokens, wich are each represented by 12 layers each comprising 768 dimensions resultas in (10 x 12 x 768) 92 160 values.