yea - an octree - se tree structure where u walk foraard and find “more detail” along the path. LMs essentially do the same, but each branching point is a token and each token has about 30.000 different possibilies - for different branches (thats the models “token vocabulary”) so yea-… “octrees”… sure exist.
huh! thought octrees could be used to search token space for the closest(s) tokens, i had never seen the word “octree” in a machine learning context.
also, i think that’s just called a “tree”, “oct-” means “eight”
also octtrees are specific to three dimensions, i did not register that and thought it was possible to apply to high dimensional spaces such as token embeddings
yea - an octree - se tree structure where u walk foraard and find “more detail” along the path. LMs essentially do the same, but each branching point is a token and each token has about 30.000 different possibilies - for different branches (thats the models “token vocabulary”) so yea-… “octrees”… sure exist.
anyway - have a nice day ~ ~ ~<3
huh! thought octrees could be used to search token space for the closest(s) tokens, i had never seen the word “octree” in a machine learning context. also, i think that’s just called a “tree”, “oct-” means “eight”
oooooooh ir right. sili meeeeeh
also octtrees are specific to three dimensions, i did not register that and thought it was possible to apply to high dimensional spaces such as token embeddings