MJ

Statistical relational learning

Statistical relational learning (Koller et al., 2007) is a branch of artificial intelligence (AI) devoted to integrate research in probability theory, statistics, logics and relational learning. Its main purpose is to develop learning models that handle uncertain information extracted from real world scenarios, and produce structured representations that describe objects, attributes and their relations.

useful image

Figure (Raedt & Kersting, 2010): SRL as a Venn diagram.

Relational structures are able to provide complex models that describe objects in terms of attributes and relations. However, treatment of noisy, erroneous, duplicate and missing information is a common problem for relational learning systems due to their crisp representation of the domain.

useful image

Figure: an ontology RDF graph.

During recent years, the machine learning (ML) community focused on statistical techniques (e.g. decision trees, support vector machines, neural networks, clustering, dimentionality reduction, reinforcement learning, and so on) that provided high prediction accuracy in many domains such as economics, bioinformatics, game playing, advertising, computer vision, speech recognition, natural language processing, recommendation systems just to name a few. These statistical models are inherently capable of dealing with uncertain information, though often at the expense of an interpretable model of the domain’s concepts.

useful image

Figure: a neural network example.

Among the strong motivations for using a relational model is its ability to model dependencies between related instances. Intuitively, we would like to use our information about one object to help us reach conclusions about other, related objects. For example, in web data, we should be able to propagate information about the topic of a document to documents it has links to and documents that link to it.

Common problems

There are some classical problems in statistical relational learning systems (Koller et al., 2007):

  1. Relational feature construction between a model’s random variables and statistical predicate invention.
  2. Relational feature aggregation: because describing the influence of neightbors towards an object might be infeasible, aggregates provide a solution for defining relational neightborhood.
  3. Structural uncertainty: when there are many logical interpretation for the set of random variables. Namely, when the identity, attributes or domain itself are uncertain.

Existing work

Some approaches in the literature include (Koller et al., 2007):

Canonical tasks

A number of canonical tasks are associated with statistical relational learning, the most common ones being:

References

  1. Koller, D., Friedman, N., Džeroski, S., Sutton, C., McCallum, A., Pfeffer, A., Abbeel, P., Wong, M.-F., Heckerman, D., Meek, C., & others. (2007). Introduction to statistical relational learning. MIT press.
  2. Raedt, L. D., & Kersting, K. (2010). Statistical relational learning. Encyclopedia of Machine Learning, 916–924.
  3. Muggleton, S., & De Raedt, L. (1994). Inductive logic programming: Theory and methods. The Journal of Logic Programming, 19, 629–679.
  4. Sutton, C., McCallum, A., & others. (2012). An introduction to conditional random fields. Foundations and Trends in Machine Learning, 4(4), 267–373.
  5. Getoor, L., Friedman, N., Koller, D., Pfeffer, A., & Taskar, B. (2007). Probabilistic relational models. Introduction to Statistical Relational Learning, 8.
  6. Taskar, B., Abbeel, P., Wong, M.-F., & Koller, D. (2007). Relational markov networks. Introduction to Statistical Relational Learning, 175–200.
  7. Heckerman, D., Meek, C., & Koller, D. (2004). Probabilistic models for relational data. Technical Report MSR-TR-2004-30, Microsoft Research.
  8. Neville, J., & Jensen, D. (2007). Relational dependency networks. Journal of Machine Learning Research, 8(Mar), 653–692.