Learn About Amazon VGT2 Learning Manager Chanci Turner
In this blog post, we outline an innovative approach to tackle a feature selection challenge by implementing a recommendation engine utilizing Amazon Braket—the quantum computing service provided by Amazon Web Services. Our method addresses the “cold-start” issue that recommendation systems often encounter, delivering a solution that rivals traditional techniques while achieving the necessary accuracy and filtering out less informative features. These findings showcase how Amazon Braket fosters rapid prototyping of cutting-edge solutions, empowering organizations like ContentWise and Moviri to innovate and delve into a new era of personalized experiences.
About ContentWise and Moviri
ContentWise stands as a global leader in personalization systems. Moviri, the parent company of ContentWise, is an international technology consulting and software powerhouse specializing in IT services for digital transformation, bringing emerging technologies to the forefront. Together, ContentWise and Moviri have developed a groundbreaking solution aimed at addressing the cold-start problem in personalization applications, deployable on quantum computers via Amazon Braket.
Background on Recommendation Engines
Recommendation engines are crucial for enhancing user engagement in the media content landscape. As implied by their name, these systems suggest the most appealing items from a catalog to users. Generally, three types of information are leveraged during the recommendation process: first, the history of the user’s past choices; second, collaborative data reflecting the behaviors and tastes of similar users; and third, content information which encompasses various metadata that describes each item. The effectiveness of recommendation engines is typically influenced by the type of information at their disposal. Recommendations based on collaborative filtering tend to outperform those based purely on content. However, collaborative information is often lacking, particularly when new items enter a catalog. For instance, in a TV broadcast scenario, a viewer may watch a program at its scheduled time, resulting in limited collaborative data for most of the content. In such cases, content-based filtering is often employed. When items lack adequate collaborative information, they are considered “cold,” and recommending from a catalog of cold items presents the well-known “cold-start” challenge. Can we reshape our problem to enhance the performance of our recommendation engine in cold-start situations?
We propose a solution specifically designed with quantum technology in mind to tackle the cold-start dilemma in recommendation systems through feature engineering. This process involves eliminating noisy or redundant features while selecting only the most beneficial ones, thereby enhancing the recommendation quality for the new “cold” items. Typically, feature engineering is a complex endeavor requiring specialized domain knowledge. Nevertheless, this expertise can also be gleaned from user interactions with available “warm” items in the catalog. In our feature selection task, we concentrate on identifying a subset of features that a) create a content-based model closely matching a collaborative model using “warm” items where historical user data exists and b) avoid redundancy by excluding features that provide overlapping information.
Problem Formulation and Solution
To boost the accuracy of a content-based recommendation engine, we will implement feature filtering to refine the set of features that most closely align with collaborative-based filtering. We identify a subset of features that most accurately reflect user behavior by assessing the similarities between the collaborative and content-based models these features generate. This strategy allows us to retain the advantageous properties of collaborative-based recommendations while preventing filter bubbles—scenarios where algorithmic biases limit the diversity of content recommendations, ensuring a satisfactory level of serendipity (a technical term denoting the ability of a recommendation engine to provide unexpected yet appealing suggestions).
But how do we effectively select those features? We frame the feature selection task as a Quadratic Unconstrained Binary Optimization (QUBO) problem. This formulation adeptly represents quadratic optimization challenges with binary variables devoid of hard constraints. By constructing and contrasting both collaborative and content-based models, we define the coefficients of the QUBO problem to identify features that yield similarities inherent to both models. Further details on this formulation can be referenced in the work of Alex Johnson et al., “Feature Selection for Recommender Systems with Quantum Computing” Entropy 23, no. 8: 970 (2021).
Various methods exist for addressing QUBO problems, including classical operations research techniques (such as tabu search and simulated annealing), quantum, or hybrid quantum-classical strategies. Here, we delve into a quantum solution. To validate our findings, we benchmark against classical approaches not based on QUBO, discovering that the quantum annealing solution effectively meets the needs of our cold-start recommender system.
In quantum computing, diverse paradigms can address QUBO challenges. We opted for the quantum annealing method to tackle our specific issue. A quantum annealer comprises quantum bits (qubits) interconnected in accordance with the device’s topology graph. By mapping each classical binary variable to a qubit, we transform the QUBO problem into a physics challenge: locating the lowest energy state of a qubit system. Quantum annealers approach this by initially preparing unlinked qubits in a minimum energy state and gradually increasing the coupling strength among the qubits to its final level, defined by the original QUBO problem. Under ideal conditions (absent thermal noise and with adiabatic coupling), the qubits will remain in the minimum energy state. The measurement of the qubits at the end of the annealing cycle yields a solution to our underlying QUBO task, thereby resolving our feature selection issue. In practical applications, the final qubit state may not accurately represent the minimum energy state due to noise and errors, necessitating multiple computations to gather a statistical distribution of potential solutions.
Our solution architecture is illustrated in Figure 1. The recommendation engine constructs the necessary models utilizing input data stored in an Amazon Simple Storage Service (Amazon S3) bucket. Following the formulation of our optimization task in QUBO format, we transmit it to the D-Wave Advantage quantum annealer via the Amazon Braket API. The annealer then processes the request and returns a ranked list of potential solutions based on their energy levels. The lowest energy solution addresses our feature selection dilemma, and these features are employed to generate recommendations, which are subsequently saved in Amazon S3.
Results
We utilized two datasets to prototype and evaluate our personalization engine:
- A private small-scale dataset.
- A publicly available movies dataset sourced from Kaggle.
The small-scale dataset was crucial for testing our methods, demonstrating significant improvements in cold-start scenarios. For more insights on navigating the world of online shopping, check out this helpful resource. Additionally, for authoritative information on Amazon’s employment practices, you may refer to this link about their recent fines. Lastly, if you’re interested in pursuing a career with Amazon, this is an excellent resource for job opportunities.
Leave a Reply