![\"\"](\"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/06\/pexels-henry-co-2250136-1024x683.jpg\")
<\/p>\n<\/p>\n
In this blog post, you\u2019ll learn about some examples of decision processes you can use in recommender systems: do you see any usage for recommending less popular products as a way to improve your business? You will see it now!<\/p>\n
Let\u2019s imagine a use case where you are building a MOOC platform (like Udemy\/Coursera). Your CEO means to increase student engagement and recurrence<\/b> in the platform, and wants to know the preferred courses for each student<\/b>.<\/p>\n
Your team of Data Scientists, after brainstorming, decides you should build a model which predicts the affinity each user has for each course<\/b>, and build a decision process on top of it. Your marketing team is able to send a fixed amount of newsletters per week <\/b>with limited slots, of the best courses for each user, at no associated cost. MOOCs websites, being online services, can easily generate a lot of user data with no major acquisition effort. For this example, the most relevant characteristics are:<\/p>\n The focus of this blog post is not to explore data sources, so let\u2019s assume we have a model that takes in this data.<\/p>\n There are three major groups of models for recommendation systems: content-based, collaborative filtering, and hybrid. At NILG.AI, we have built content-based algorithms for a lot of our clients, as they allow you to compare products by their characteristics, reducing cold start in new products.<\/p>\n For more details when comparing these types of models, we recommend an in-depth article such as this one<\/a>.<\/p>\n Now towards the main topic of the blog post: how do you go from a list of affinities per user to an actual recommendation in the newsletters? There are multiple options for creating the final newsletter:<\/b><\/p>\n There is no actual best rule for this: one approach we have followed in the past was to run a \u00a0hybrid<\/b> approach, where you combine sections of the newsletter using different strategies. You need to test out these proportions in different AB Tests.<\/p>\n So\u2026 it does make sense that you can end up recommending less popular products to improve your business: you are giving exposure to less popular courses, preventing them from ending up in a feedback loop. If the courses are rarely recommended, they don\u2019t get a lot of sales\/clicks. If they don\u2019t get a lot of sales\/clicks, they are rarely recommended!<\/p>\n To measure if the model is doing well, besides the technical metrics (Lift, ROC AUC, Average Precision, \u2026) there are some business KPIs you can measure. These KPIs, for these types of businesses, are typically based on positive feedback signals, such as click rates and sales. Some are more short-term but lead to a less relevant intent signal (such as clicks).<\/p>\n For instance, you can focus on:<\/p>\n Take into consideration that, while some KPIs might increase, others might decrease: for instance, you can dramatically increase course exposure rate but decrease the overall experience: this would happen if you randomly recommend a course, for instance.<\/p>\n Using strategies such as Student to Course recommendations, you should be improving your business-centric KPIs and decreasing your user-centric KPIs, when compared to a baseline. On the other hand, Course to Student is an approach that is more user-centric. There\u2019s always a trade-off you need to consider and discuss with the business teams.<\/p>\n Regarding AB Testing Strategies: you will need to experiment with the proportion type of each strategy if you want to use the hybrid strategy as mentioned above. The number of groups should be dependent on the number of active users you have and the time you want to run these experiments for.<\/p>\n Note that in many cases, there\u2019s a budget <\/b>associated with course recommendation (for instance, you can only recommend N courses per week, due to a maximum number of sales, caused by digital license issues). In this case, AB Testing can lead to unreliable conclusions due to cannibalization bias<\/a>:\u00a0 the recommendations in one strategy are influencing the others, breaking the rules of standard AB Testing. We\u2019ll talk more about this later in another blog post!<\/p>\n There\u2019s a lot to be done after having a prediction of how much a student likes a certain product. It\u2019s up to the Data Scientists to help define these constraints, together with the rest of the team. With a huge diversity of approaches to follow, there\u2019s really not a \u201cbest solution\u201d. You should quickly iterate to generate value, while continuously running R&D activities and monitoring the performance of the current approach. Is this the solution you need? If so, contact us<\/a>. Otherwise, contact us<\/a> anyway, and let\u2019s find a better solution together!<\/p>","protected":false},"excerpt":{"rendered":" In this blog post, you\u2019ll learn about some examples of decision processes you can use in recommender systems: do you see any usage for recommending less popular products as a way to improve your business? You will see it now! The Use Case Let\u2019s imagine a use case where you are building a MOOC platform […]<\/p>\n","protected":false},"author":5,"featured_media":1482,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[84],"tags":[48,46,89],"class_list":["post-1480","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-use-case","tag-ai4tech","tag-marketing","tag-recommender-system"],"acf":[],"yoast_head":"\n![\"\"](\"https:\/\/nilg.ai\/wp-content\/uploads\/2022\/05\/Figure-1.svg\")
<\/a><\/p>\nData<\/b><\/h2>\n
\n
Model<\/b><\/h2>\n
Decision Process<\/b><\/h2>\n
Rule-based approaches:\u00a0<\/b><\/h4>\n
\n
Recommend Top-K courses to each student (Course to Student).<\/b><\/h4>\n
\n
Recommend Top-K students to each course (Student to Course).<\/b><\/h4>\n
\n
Urgency trade-off of individual vs. overall affinity<\/b><\/h4>\n
\n
Randomly recommend courses<\/b><\/h4>\n
\n
<\/a><\/p>\nModel Evaluation<\/b><\/h2>\n
\n
AB Testing<\/b><\/h2>\n
Conclusions<\/b><\/h2>\n
\nRemember that before jumping into the complexities of building personalized recommendations, it’s equally crucial to have a solid eCommerce. We recommend our partner Pipecodes<\/a> in case you’re looking to improve your eCommerce.<\/p>\n