{"id":2518,"date":"2022-09-22T15:37:40","date_gmt":"2022-09-22T15:37:40","guid":{"rendered":"https:\/\/nilg.ai\/?p=2518"},"modified":"2023-09-01T18:33:04","modified_gmt":"2023-09-01T18:33:04","slug":"turning-classes-into-inputs","status":"publish","type":"post","link":"https:\/\/nilg.ai\/pt\/202209\/turning-classes-into-inputs\/","title":{"rendered":"Turning classes into inputs"},"content":{"rendered":"

\"\"<\/p>\n

Let\u2019s face it, we all have worked on an ML project where we had to predict a ridiculously high number of classes. Large enough to make the number of observations per class into an embarrassingly small subset. Most people model these tasks as a multiclass classification problem where, for each input observation, we must predict the most likely class (or the class probabilities).<\/span><\/p>\n

Examples of such tasks are predicting the model of a car, the species of an animal, the intent of a user on a chat, the SIC\/NAICS<\/a> code of a company, and the product on a marketplace picture, among many others.<\/span><\/p>\n

A dynamic number of classes also characterizes these examples. For example, let\u2019s say we are training a Computer Vision<\/a> model to recognize the item on a photo for an autonomous retail store. Every day, new products are launched to the market. If you go with the traditional approach, you must train a new model daily to keep up with the catalog.<\/span><\/p>\n

This would make the model maintenance (and operations) go wild! You don\u2019t want that!<\/span><\/p>\n

Our recipe for cooking large Multiclass classification models<\/span><\/h2>\n

Our trick for this kind of model is converting classes into part of the question. So, instead of training a multiclass classification model that predicts:<\/span><\/p>\n

What\u2019s the class of the observation? \u2013\u00a0 <\/b>a categorical question<\/p>\n

we ask the question:<\/span><\/p>\n

Is this observation from a given category?<\/strong> \u2013 a yes or no question.<\/p>\n

 <\/p>\n

I like to call this trick flipping your model upside down, making the outcome part of the inputs.<\/p><\/blockquote>\n

 <\/p>\n

Technically, we transform our predictive model\u00a0<\/span><\/p>\n

  <\/span>   <\/span>\"$$F :: observation \rightarrow class$$\"<\/p>\n

into\u00a0<\/span><\/p>\n

  <\/span>   <\/span>\"$$F :: observation x class \rightarrow yes/no$$\"<\/p>\n

Then, for any given observation, you just need to ask for all classes and take the one with the highest probability.<\/span><\/p>\n

  <\/span>   <\/span>\"$$argmax_{class} F(observation, class)$$\"<\/p>\n

Is there a new class? Don\u2019t worry; just ask an additional question next time you need to generate a prediction. <\/span>No re-training is required<\/b>. I like to call this trick flipping your model upside down, making the outcome part of the inputs.<\/span><\/p>\n

\"\"<\/a><\/p>\n

 <\/p>\n

Disclaimer:<\/b> as long as your initial class subset is general enough. Otherwise, just re-train every now and then.<\/span><\/p>\n

How can we encode classes as inputs? Multiclass classification as Binary classification\u00a0<\/span><\/h2>\n

Let\u2019s say you have features describing the classes. Then, you just need to encode the class as the set of features that describe it. For example, in the retail product recognition example, you can characterize the item by its category, brand, weight, size, color, description, ingredients, etc.<\/span><\/p>\n

<\/p>\n

  <\/span>   <\/span>\"$$F :: observation \times classFeatures \rightarrow yes/no$$\"<\/p>\n

<\/span><\/p>\n

\"\"<\/a><\/p>\n

However, it\u2019s not so common to have features describing the classes. How would you describe a user’s intent on a chat? How would you describe a car model or an animal?<\/span><\/p>\n

Yes, it would be possible to do it. But, my bet is that you won\u2019t have access to such data.<\/span><\/p>\n

What to do in such a situation?<\/span><\/p>\n

A Card Up Your Sleeve<\/span><\/h3>\n

You must agree that you have the features of the entities belonging to that class, right? In that case, you can just get features about the distribution of the observations in that class. Statistical values like the average, minimum, maximum, and variance of the features for the observations in that class. Now you have features describing the class. You\u2019re welcome.<\/span><\/p>\n

Hey Kelwin, but you know, aren’t features old-fashioned? We all work with deep learning nowadays and leave the model to learn its own features. I\u2019m glad you asked, young grasshopper!<\/span><\/p>\n \n\n

\n\t\t
\n\t\t \n\t\t \"Kelwin\t\t <\/div>\n\n
\n\t

Do you want to further discuss this idea?<\/h3>

Book a meeting with Kelwin Fernandes<\/strong><\/p>\tMeet Kelwin<\/a>\n\t\t\t\n\tSaber mais<\/a>\n\t\t\t<\/div>\n\t<\/div>\n\n

You can train a siamese neural network that answers the question:<\/span><\/p>\n

Are these two observations from the same class?<\/strong><\/p>\n

 <\/p>\n

Or, in a more formal language:<\/span><\/p>\n

<\/p>\n

  <\/span>   <\/span>\"$$F :: observation_1 \times observation_2 \rightarrow yes/no$$\"<\/p>\n

<\/span><\/p>\n

\"\"<\/a><\/p>\n

 <\/p>\n

Now, you can ask the question comparing your new test observation against all training data points, aggregate the probabilities by class (e.g., maximum, average) and return the class with the highest score. Basically, you can just transform a multiclass problem into a similarity learning one.<\/span><\/p>\n

Are you crazy?<\/b> That won\u2019t scale at all. Well, it will. First of all, you just need to index all training observations. So, whenever new input arrives, you just run your neural network on the input instance to get its latent features plus a simple nearest neighbor comparison against all the other data points.<\/span><\/p>\n

Still, can you imagine doing that over millions of observations?<\/strong> Of course not, but you can always choose pivots that represent your class properly using any technique, such as k-medoids on the latent space. Easy peasy.<\/span><\/p>\n

Now, you have <\/span>a scalable model that adjusts to new classes without the need for re-training<\/b>.<\/span><\/p>\n

We have used this trick in several industries and use cases, which always pays for itself.<\/span><\/p>\n

\n\t\t
\"\"<\/div>\n\t\t
Curso<\/h6>

O Espectro do Machine Learning<\/h3>\n\t\t\t

Learn more tricks like this in our ML course.<\/p>\n\t\t\tSaber mais<\/a>\n\t\t<\/div>\n\t<\/div>\n

You gain so much operational efficiency, plus mitigating the problem of classes with low frequency.<\/span><\/p>\n

Is a class no longer relevant?<\/strong> Remove its observations from your index.<\/span><\/p>\n

Is there any new class?<\/strong> Add new observations to your index.<\/span><\/p>\n

As easy as that!<\/span><\/p>\n

 <\/p>\n

There are a couple of additional tricks we can teach you, but you will need to wait for another article. I have to leave. But you don\u2019t. So, subscribe now to our newsletter below to stay tuned.<\/span><\/p>","protected":false},"excerpt":{"rendered":"

Let\u2019s face it, we all have worked on an ML project where we had to predict a ridiculously high number of classes. Large enough to make the number of observations per class into an embarrassingly small subset. Most people model these tasks as a multiclass classification problem where, for each input observation, we must predict […]<\/p>\n","protected":false},"author":2,"featured_media":2543,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[53],"tags":[48,81,45],"class_list":["post-2518","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technical","tag-ai4tech","tag-deep-learning","tag-machine-learning"],"acf":[],"yoast_head":"\nTurning classes into inputs - NILG.AI<\/title>\n<meta name=\"description\" content=\"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nilg.ai\/pt\/202209\/turning-classes-into-inputs\/\" \/>\n<meta property=\"og:locale\" content=\"pt_PT\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Turning classes into inputs - NILG.AI\" \/>\n<meta property=\"og:description\" content=\"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nilg.ai\/pt\/202209\/turning-classes-into-inputs\/\" \/>\n<meta property=\"og:site_name\" content=\"NILG.AI\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-22T15:37:40+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-01T18:33:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/09\/girl-lies-field-with-her-legs-up-sandals-funny-foot-background-field-sky.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1900\" \/>\n\t<meta property=\"og:image:height\" content=\"1268\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Kelwin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@nilg_ai\" \/>\n<meta name=\"twitter:site\" content=\"@nilg_ai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kelwin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutos\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/\"},\"author\":{\"name\":\"Kelwin\",\"@id\":\"https:\/\/nilg.ai\/#\/schema\/person\/459afa4b4816694ebca008f362e65466\"},\"headline\":\"Turning classes into inputs\",\"datePublished\":\"2022-09-22T15:37:40+00:00\",\"dateModified\":\"2023-09-01T18:33:04+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/\"},\"wordCount\":904,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/nilg.ai\/#organization\"},\"keywords\":[\"AI4tech\",\"Deep Learning\",\"Machine Learning\"],\"articleSection\":[\"Technical\"],\"inLanguage\":\"pt-PT\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/\",\"url\":\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/\",\"name\":\"Turning classes into inputs - NILG.AI\",\"isPartOf\":{\"@id\":\"https:\/\/nilg.ai\/#website\"},\"datePublished\":\"2022-09-22T15:37:40+00:00\",\"dateModified\":\"2023-09-01T18:33:04+00:00\",\"description\":\"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.\",\"inLanguage\":\"pt-PT\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nilg.ai\/#website\",\"url\":\"https:\/\/nilg.ai\/\",\"name\":\"NILG.AI\",\"description\":\"Create ever-improving businesses with AI\",\"publisher\":{\"@id\":\"https:\/\/nilg.ai\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nilg.ai\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"pt-PT\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nilg.ai\/#organization\",\"name\":\"NILG.AI\",\"url\":\"https:\/\/nilg.ai\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"pt-PT\",\"@id\":\"https:\/\/nilg.ai\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/03\/logo.svg\",\"contentUrl\":\"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/03\/logo.svg\",\"caption\":\"NILG.AI\"},\"image\":{\"@id\":\"https:\/\/nilg.ai\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/twitter.com\/nilg_ai\",\"https:\/\/youtube.com\/@nilg_ai\",\"https:\/\/www.linkedin.com\/company\/nilg-ai\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/nilg.ai\/#\/schema\/person\/459afa4b4816694ebca008f362e65466\",\"name\":\"Kelwin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"pt-PT\",\"@id\":\"https:\/\/nilg.ai\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4dfbc723a2ec524bcc39be19be394d4278c06e540c818521e644d8f120a2bbfd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4dfbc723a2ec524bcc39be19be394d4278c06e540c818521e644d8f120a2bbfd?s=96&d=mm&r=g\",\"caption\":\"Kelwin\"},\"url\":\"https:\/\/nilg.ai\/pt\/author\/kelwin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Turning classes into inputs - NILG.AI","description":"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nilg.ai\/pt\/202209\/turning-classes-into-inputs\/","og_locale":"pt_PT","og_type":"article","og_title":"Turning classes into inputs - NILG.AI","og_description":"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.","og_url":"https:\/\/nilg.ai\/pt\/202209\/turning-classes-into-inputs\/","og_site_name":"NILG.AI","article_published_time":"2022-09-22T15:37:40+00:00","article_modified_time":"2023-09-01T18:33:04+00:00","og_image":[{"width":1900,"height":1268,"url":"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/09\/girl-lies-field-with-her-legs-up-sandals-funny-foot-background-field-sky.jpg","type":"image\/jpeg"}],"author":"Kelwin","twitter_card":"summary_large_image","twitter_creator":"@nilg_ai","twitter_site":"@nilg_ai","twitter_misc":{"Written by":"Kelwin","Est. reading time":"5 minutos"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/#article","isPartOf":{"@id":"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/"},"author":{"name":"Kelwin","@id":"https:\/\/nilg.ai\/#\/schema\/person\/459afa4b4816694ebca008f362e65466"},"headline":"Turning classes into inputs","datePublished":"2022-09-22T15:37:40+00:00","dateModified":"2023-09-01T18:33:04+00:00","mainEntityOfPage":{"@id":"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/"},"wordCount":904,"commentCount":0,"publisher":{"@id":"https:\/\/nilg.ai\/#organization"},"keywords":["AI4tech","Deep Learning","Machine Learning"],"articleSection":["Technical"],"inLanguage":"pt-PT","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/","url":"https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/","name":"Turning classes into inputs - NILG.AI","isPartOf":{"@id":"https:\/\/nilg.ai\/#website"},"datePublished":"2022-09-22T15:37:40+00:00","dateModified":"2023-09-01T18:33:04+00:00","description":"An easy trick to train scalable multiclass classification Machine Learning models by transforming classes into input data.","inLanguage":"pt-PT","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nilg.ai\/202209\/turning-classes-into-inputs\/"]}]},{"@type":"WebSite","@id":"https:\/\/nilg.ai\/#website","url":"https:\/\/nilg.ai\/","name":"NILG.AI","description":"Create ever-improving businesses with AI","publisher":{"@id":"https:\/\/nilg.ai\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nilg.ai\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"pt-PT"},{"@type":"Organization","@id":"https:\/\/nilg.ai\/#organization","name":"NILG.AI","url":"https:\/\/nilg.ai\/","logo":{"@type":"ImageObject","inLanguage":"pt-PT","@id":"https:\/\/nilg.ai\/#\/schema\/logo\/image\/","url":"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/03\/logo.svg","contentUrl":"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/03\/logo.svg","caption":"NILG.AI"},"image":{"@id":"https:\/\/nilg.ai\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/twitter.com\/nilg_ai","https:\/\/youtube.com\/@nilg_ai","https:\/\/www.linkedin.com\/company\/nilg-ai\/"]},{"@type":"Person","@id":"https:\/\/nilg.ai\/#\/schema\/person\/459afa4b4816694ebca008f362e65466","name":"Kelwin","image":{"@type":"ImageObject","inLanguage":"pt-PT","@id":"https:\/\/nilg.ai\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/4dfbc723a2ec524bcc39be19be394d4278c06e540c818521e644d8f120a2bbfd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4dfbc723a2ec524bcc39be19be394d4278c06e540c818521e644d8f120a2bbfd?s=96&d=mm&r=g","caption":"Kelwin"},"url":"https:\/\/nilg.ai\/pt\/author\/kelwin\/"}]}},"_links":{"self":[{"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/posts\/2518","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/comments?post=2518"}],"version-history":[{"count":10,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/posts\/2518\/revisions"}],"predecessor-version":[{"id":3424,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/posts\/2518\/revisions\/3424"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/media\/2543"}],"wp:attachment":[{"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/media?parent=2518"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/categories?post=2518"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nilg.ai\/pt\/wp-json\/wp\/v2\/tags?post=2518"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}