If one term has captured the popular imagination of late, it’s artificial intelligence (AI). Depending on whom you ask, it’s either the panacea for humanity’s woes, or a harbinger of the end times. And yet, how many of us can truly say that we understand AI? Can either the optimists or the pessimists actually justify their positions? What does the AI future actually look like?

• ai
• future

At its core, decision-making is about answering questions. Should we launch this product? Should I order take-out for dinner? Should I go on this date? These are conditional questions that reflect some number of possible futures. The decision occurs when you select one of the futures and take the action that it prescribes.

• decision making
• questions

Search relevance is a difficult problem in information retrieval. How do you ensure that you get the best results back from searching a collection of documents? Let's explore a few basic strategies, including simple searching, term-frequency searching, and TF-IDF searching.

• information retrieval
• nlp
• search

Term frequency-inverse document frequency is a means of assigning weight to a search term when comparing individual documents within a corpus. It is an improvement on the bag-of-words model in that it considers the relative rarity of a term within a larger corpus.

• information retrieval
• linguistics
• nlp
• reference

The bag-of-words (BoW) model is used to represent a document as a fixed-length vector of frequency counts for each token in the document. BoW models are useful in classification, machine learning, and topic modeling tasks, among others.

• information retrieval
• linguistics
• nlp
• reference

Writing is humanity’s superpower — when done well, it informs, provokes, and entertains. Perhaps that is why blogging is so popular among programmers. We’re a naturally curious community and sharing knowledge is an integral part of our ethos.

• essay
• writing

Natural language processing is a complex and evolving field. Part computer science, part linguistics, part statistics--it can be a challenge deciding where to begin. One starting place is to look at the most influential papers in academic literature--if you can master these papers, then you'll be well on the path to becoming an NLP expert.

• academia
• ai
• books
• nlp

One of the great things about using Python for natural language processing (NLP) is the large ecosystem of tools and libraries. From tokenization, to machine learning, to data visualization--Python has something for every NLP task in your workflow. Of course, choosing the right tool isn't always so easy.

• deep learning
• machine learning
• nlp
• python
• tools

In mid-August 2020, the Stanford NLP Group released version 1.1.1 of the Stanza Python library. Let's explore what Stanza can do, and while we're at it, see if we can learn something about the characters in a classic novel.

• literature
• nlp
• python
• stanza
• text analysis

In natural language processing, stemming is the process of reducing a word to its stem form. Typically, stemming is used as part of an NLP pipeline in order to reduce all words in a text to their stems so that they can be analyzed together.

• linguistics
• nlp
• nltk
• reference