Historical Context
Back in 1982, Japan launched an offensive in becoming the global AI superpower. It founded a 10-year initiative called the “japanese 5th generation computing project“, which was revolutionary but failed. Their idea was to use a large number of CPUs, which where at the time much less powerful, however, nowadays AI has the same foundation, as NVIDIA’s GPU are nothing else than many, many simplified CPUs where each core on its own is much less capable, but the whole of it is super powerful.
Apart from the CPU assumption, the members of this project assumed not to use the probabilistic-based models commonly called neural networks (or next-token-prediction models). So this other approach was using logic instead of probability, which was so forth not yet implemented to create as powerful AI as the current best AI models (which are transformer-based models, such as GPT and deepseek).
Douglas Lenat, founder of the Cyc project, says in the youtube clip below, that the ultimately failure of the Japanese project originated in the problem of trying to arranging all kind of things, knowledge and information into a hierarchical representation, while actually there is indications that this is not possible. In his words, in the youtube clip below: “Why the Japanese fifth generation Computing effort failed, there were about half a dozen different reasons. One of the reasons they failed, was because they tried to represent knowledge as a tree, rather than as a graph.
Douglas Lenat embarked on a logic-based machine learning project in 1984, sharing his first results in the publication of CYC in 1985. The first stage of the Cyc project’s development was accomplished through extensive manual axiom-writing, and eventually evolved into OpenCyc 4.0, which was released in June 2012. However, in 2017, the project shut down, but Douglas Lenate continues this research and development within the Cyc-company.
1st paragraph: topic
Popularity-based, keyword, and semantic searches, while useful for everyday tasks, have significant limitations in scientific research, where researchers need highly specific details that are often mentioned only tangentially in documents. To address this challenge, Information Extraction (IE) presents two promising solutions: applying IE after the search to reduce manual skimming or leveraging IE beforehand to generate data structure that enhances search efficiency. Traditional search techniques still constrain the first method, while the second, by building a knowledge graph from extracted data, allows researchers to bypass documents and directly target information.
2nd paragraph: gap
However, state-of-the-art IE models predominantly rely on supervised fine-tuning of large language models (LLMs), which derive their effectiveness from the quality and scope of their training data. Creating such datasets is resource-intensive, which lead to datasets that focus on extracting details relevant only to narrow topics, while discarding information that falls outside this regiment. This approach introduces a significant limitation: each dataset represents a distinct extraction task, with varying annotation schemes that differ across datasets. The transition to applications carried these disadvantages with it and as a result, information fragmented across separate databases, each structured by its own unique schema and ontology.
The relevant literature for text-to-graph translation is best divided respective two elements of this project. Please navigate to the one that is more exciting to you:
- Information extraction Literature
- Universal Knowledge Representation Literature