Word error rate

Word error rate

Word error rate (WER) is a common metric of the performance of a speech recognition or machine translation system. The WER metric typically ranges from 0 to 1, where 0 indicates that the compared pieces of text are exactly identical, and 1 (or larger) indicates that they are completely different with no similarity. This way, a WER of 0.8 means that there is an 80% error rate for compared sentences. The general difficulty of measuring performance lies in the fact that the recognized word sequence can have a different length from the reference word sequence (supposedly the correct one). The WER is derived from the Levenshtein distance, working at the word level instead of the phoneme level. The WER is a valuable tool for comparing different systems as well as for evaluating improvements within one system. This kind of measurement, however, provides no details on the nature of translation errors and further work is therefore required to identify the main source(s) of error and to focus any research effort. This problem is solved by first aligning the recognized word sequence with the reference (spoken) word sequence using dynamic string alignment. Examination of this issue is seen through a theory called the power law that states the correlation between perplexity and word error rate. Word error rate can then be computed as: W E R = S + D + I N = S + D + I S + D + C {\displaystyle {\mathit {WER}}={\frac {S+D+I}{N}}={\frac {S+D+I}{S+D+C}}} where S is the number of substitutions, D is the number of deletions, I is the number of insertions, C is the number of correct words, N is the number of words in the reference (N=S+D+C) The intuition behind 'deletion' and 'insertion' is how to get from the reference to the hypothesis. So if we have the reference "This is wikipedia" and hypothesis "This _ wikipedia", we call it a deletion. Note that since N is the number of words in the reference, the word error rate can be larger than 1.0, namely if the number of insertions I is larger than the number of correct words C. When reporting the performance of a speech recognition system, sometimes word accuracy (WAcc) is used instead: W A c c = 1 − W E R = N − S − D − I N = C − I N {\displaystyle {\mathit {WAcc}}=1-{\mathit {WER}}={\frac {N-S-D-I}{N}}={\frac {C-I}{N}}} Since the WER can be larger than 1.0, the word accuracy can be smaller than 0.0. == Experiments == It is commonly believed that a lower word error rate shows superior accuracy in recognition of speech, compared with a higher word error rate. However, at least one study has shown that this may not be true. In a Microsoft Research experiment, it was shown that, if people were trained under "that matches the optimization objective for understanding", (Wang, Acero and Chelba, 2003) they would show a higher accuracy in understanding of language than other people who demonstrated a lower word error rate, showing that true understanding of spoken language relies on more than just high word recognition accuracy. == Other metrics == One problem with using a generic formula such as the one above, however, is that no account is taken of the effect that different types of error may have on the likelihood of successful outcome, e.g. some errors may be more disruptive than others and some may be corrected more easily than others. These factors are likely to be specific to the syntax being tested. A further problem is that, even with the best alignment, the formula cannot distinguish a substitution error from a combined deletion plus insertion error. Hunt (1990) has proposed the use of a weighted measure of performance accuracy where errors of substitution are weighted at unity but errors of deletion and insertion are both weighted only at 0.5, thus: W E R = S + 0.5 D + 0.5 I N {\displaystyle {\mathit {WER}}={\frac {S+0.5D+0.5I}{N}}} There is some debate, however, as to whether Hunt's formula may properly be used to assess the performance of a single system, as it was developed as a means of comparing more fairly competing candidate systems. A further complication is added by whether a given syntax allows for error correction and, if it does, how easy that process is for the user. There is thus some merit to the argument that performance metrics should be developed to suit the particular system being measured. Whichever metric is used, however, one major theoretical problem in assessing the performance of a system is deciding whether a word has been “mis-pronounced,” i.e. does the fault lie with the user or with the recogniser. This may be particularly relevant in a system which is designed to cope with non-native speakers of a given language or with strong regional accents. The pace at which words should be spoken during the measurement process is also a source of variability between subjects, as is the need for subjects to rest or take a breath. All such factors may need to be controlled in some way. For text dictation it is generally agreed that performance accuracy at a rate below 95% is not acceptable, but this again may be syntax and/or domain specific, e.g. whether there is time pressure on users to complete the task, whether there are alternative methods of completion, and so on. The term "Single Word Error Rate" is sometimes referred to as the percentage of incorrect recognitions for each different word in the system vocabulary. == Edit distance == The word error rate may also be referred to as the length normalized edit distance. The normalized edit distance between X and Y, d( X, Y ) is defined as the minimum of W( P ) / L ( P ), where P is an editing path between X and Y, W ( P ) is the sum of the weights of the elementary edit operations of P, and L(P) is the number of these operations (length of P).

Shaded Picture System

The Shaded Picture System was a 3D raster computer display processor introduced by Evans & Sutherland in October 1973. The Shaded Picture System was the first general-purpose, commercially available raster computer graphics display processor capable of real-time, shaded 3D graphics. It could only display black and white graphics at a resolution of 256 by 256. It was extremely expensive, and very few units were ever sold. == History == The principles of shaded, hidden-line true 3D graphics were pioneered at the University of Utah in 1967. However, this algorithm was slow and would take several minutes to produce an image. In 1970, Gary Watkins developed a FORTRAN simulator of a faster algorithm that would theoretically generate shaded 3D images in real-time, "if implemented in suitable hardware". The simulator itself was still not capable of real-time shaded 3D image rendering. Evans & Sutherland developed a functional prototype of this "suitable hardware", which was later sold as the Shaded Picture System in 1973. About a year earlier in 1972, Evans & Sutherland sold the first and only CT1 to Case Western Reserve University. The CT1, or Continuous Tone 1, was a specialized image generator, not meant as a marketable or mass-produced product. At the time, the CT1, along with G.E./NASA's upgraded Electronic Scene Generator from 1971, would have been the only real-time raster graphics systems sold to customers comparable to the Shaded Picture System, although both the CT1 and Electronic Scene Generator were intentionally produced as one-off products and specialized for the needs of their customers. The Shaded Picture System, in contrast, was intentionally marketed.In early 1975, Evans & Sutherland demonstrated a random-access video frame buffer using relatively low-cost semiconductor memory, which was much more capable than the Shaded Picture System. When interfaced with a (non-shaded) E&S Picture System, the frame buffer had a resolution of 512 by 512 in grayscale and partial color capabilities. By the end of 1975, this frame buffer was commercially available.

Quantification (machine learning)

In machine learning, quantification (variously called learning to quantify, or supervised prevalence estimation, or class prior estimation) is the task of using supervised learning in order to train models (quantifiers) that estimate the relative frequencies (also known as prevalence values) of the classes of interest in a sample of unlabelled data items. For instance, in a sample of 100,000 unlabelled tweets known to express opinions about a certain political candidate, a quantifier may be used to estimate the percentage of these tweets which belong to class `Positive' (i.e., which manifest a positive stance towards this candidate), and to do the same for classes `Neutral' and `Negative'. Quantification may also be viewed as the task of training predictors that estimate a (discrete) probability distribution, i.e., that generate a predicted distribution that approximates the unknown true distribution of the items across the classes of interest. Quantification is different from classification, since the goal of classification is to predict the class labels of individual data items, while the goal of quantification it to predict the class prevalence values of sets of data items. Quantification is also different from regression, since in regression the training data items have real-valued labels, while in quantification the training data items have class labels. It has been shown in multiple research works that performing quantification by classifying all unlabelled instances and then counting the instances that have been attributed to each class (the 'classify and count' method) usually leads to suboptimal quantification accuracy. This suboptimality may be seen as a direct consequence of 'Vapnik's principle', which states: If you possess a restricted amount of information for solving some problem, try to solve the problem directly and never solve a more general problem as an intermediate step. It is possible that the available information is sufficient for a direct solution but is insufficient for solving a more general intermediate problem. In our case, the problem to be solved directly is quantification, while the more general intermediate problem is classification. As a result of the suboptimality of the 'classify and count' method, quantification has evolved as a task in its own right, different (in goals, methods, techniques, and evaluation measures) from classification. == Quantification tasks == === Quantification tasks according to the set of classes === The main variants of quantification, according to the characteristics of the set of classes used, are: Binary quantification, corresponding to the case in which there are only n = 2 {\displaystyle n=2} classes and each data item belongs to exactly one of them; Single-label multiclass quantification, corresponding to the case in which there are n > 2 {\displaystyle n>2} classes and each data item belongs to exactly one of them; Multi-label multiclass quantification, corresponding to the case in which there are n ≥ 2 {\displaystyle n\geq 2} classes and each data item can belong to zero, one, or several classes at the same time; Ordinal quantification, corresponding to the single-label multiclass case in which a total order is defined on the set of classes. Regression quantification, a task which stands to 'standard' quantification as regression stands to classification. Strictly speaking, this task is not a quantification task as defined above (since the individual items do not have class labels but are labelled by real values), but has enough commonalities with other quantification tasks to be considered one of them. Most known quantification methods address the binary case or the single-label multiclass case, and only few of them address the multi-label, ordinal, and regression cases. Binary-only methods include the Mixture Model (MM) method, the HDy method, SVM(KLD), and SVM(Q). Methods that can deal with both the binary case and the single-label multiclass case include probabilistic classify and count (PCC), adjusted classify and count (ACC), probabilistic adjusted classify and count (PACC), the Saerens-Latinne-Decaestecker EM-based method (SLD), and KDEy. Methods for multi-label quantification include regression-based quantification (RQ) and label powerset-based quantification (LPQ). Methods for the ordinal case include ordinal versions of the above-mentioned ACC, PACC, and SLD methods, and ordinal versions of the above-mentioned HDy method. Methods for the regression case include Regress and splice and Adjusted regress and sum. === Quantification tasks according to the type of data === Several subtasks of quantification may be identified according to the type of data involved. Example such tasks are: Quantification of networked data. This task consists of performing quantification when the datapoints are members of a relation, i.e., are interlinked. As such, this task is a strict relative of collective classification. Quantification over time. This task consists of performing quantification on sets that become available in a temporal sequence, i.e., as a data stream, and finds application in contexts in which class prevalence values must be monitored over time. == Evaluation measures for quantification == Several evaluation measures can be used for evaluating the error of a quantification method. Since quantification consists of generating a predicted probability distribution that estimates a true probability distribution, these evaluation measures are ones that compare two probability distributions. Most evaluation measures for quantification belong to the class of divergences. Evaluation measures for binary quantification, single-label multiclass quantification, and multi-label quantification, are Absolute Error Squared Error Relative Absolute Error Kullback–Leibler divergence Pearson Divergence Evaluation measures for ordinal quantification are Normalized Match Distance (a particular case of the Earth Mover's Distance) Root Normalized Order-Aware Distance == Applications == Quantification is of special interest in fields such as the social sciences, epidemiology, market research, allocating resources, and ecological modelling, since these fields are inherently concerned with aggregate data. However, quantification is also useful as a building block for solving other downstream tasks, such as improving the accuracy of classifiers on out-of-distribution data, measuring classifier bias and ranker bias, and estimating the accuracy of classifiers on out-of-distribution data. == Resources == LQ 2021: the 1st International Workshop on Learning to Quantify LQ 2022: the 2nd International Workshop on Learning to Quantify LQ 2023: the 3rd International Workshop on Learning to Quantify LQ 2024: the 4th International Workshop on Learning to Quantify LQ 2025: the 5th International Workshop on Learning to Quantify LeQua 2022: the 1st Data Challenge on Learning to Quantify LeQua 2024: the 2nd Data Challenge on Learning to Quantify QuaPy: An open-source Python-based software library for quantification QuantificationLib: A Python library for quantification and prevalence estimation

Timeline of artificial intelligence risks in global finance

The following article is a broad timeline of the course of events related to artificial intelligence risks in global finance. The AI boom has led to concerns including the existential risk from artificial intelligence, as the uptake on applications of artificial intelligence increases. By late 2025, global finance and artificial intelligence were "deeply intertwined". A June 2025 Menlo Ventures report raised concerns about the sustainability of future revenue and long-term profitability of AI, given the relatively low rate of consumer monetization. == 2017 == 30 NovemberThe New York Times said that new AI reports by McKinsey & Company, the National Bureau of Economic Research, and an AI Index created by university researchers, indicated an early AI boom. The Index built on a project—"The One Hundred Year Study on Artificial Intelligence" launched in 2014. == 2018 == 2018 was a year of incremental AI growth in finance. == 2022 == The release of ChatGPT by OpenAI became the catalyst for an artificial intelligence boom that continues to remake the global economy. According to a European Central Bank report, public interest in AI increased rapidly as evidenced with rising Google searches, AI jobs, models, patents, and innovations since late 2022. At that time Europe led the US in the size of its AI workforce. == 2023 == The regulatory body, the International Monetary Fund (IMF), published their report, "Generative Artificial Intelligence in Finance: Risk Considerations", drawing attention to oversight gaps and the need for regulations. The report explores the risks posed by using generative artificial intelligence (GenAI) systems in the financial sector including "broader risks to financial stability." == 2024 == January 12 In January 2024 Bloomberg's published its list of the "Magnificent Seven" Big Tech companies on the stock market based on their strength, size and market capitalization:Apple, Microsoft, Alphabet (Google), Amazon, Meta Platforms (Facebook), Nvidia, and Tesla. 21 June During the AI boom, Nvidia became the world's most valuable company, surpassing Microsoft, as its value increased to over US$4 trillion. In 2023 and 2024, the "Magnificent Seven" stocks were the primary drivers behind the increase in equity indexes, according to Reuters. == 2025 == === January === 23 January President Donald Trump's AI policy was announced calling for United States global leadership in artificial intelligence. The Economist noted that this politic shift in which the United States seeks "global dominance" in AI includes trimming regulations and assisting in expansion of infrastructure and increase in number of AI workers. Governments of Gulf nations were also investing trillions of dollars in AI. 27 January Against the backdrop of a tech war between China and the United States over AI dominance, within days of the launch of China's free DeepSeek App, it was the most downloaded app in the United States, rising to the first place in the Apple app store. President Trump responded immediately, saying this "sudden rise" should be a "wake-up" call to the United States, and called on US companies to be more competitive. === June === 26 June In their June 2025 report, Menlo Ventures estimated that only about 3% of consumers paid for artificial intelligence-related services, representing about $USD12 billion in annual spending. This is relatively low in contrast to the massive capital expenditure by AI infrastructure companies, which raises concerns about revenue sustainability and long-term profitability. === July === 23 July The Trump administration launched the US AI Action Plan, positioning the United States in a high-stakes technological race with China for global dominance in artificial intelligence, emphasizing that neither nation can afford to fall behind due to the exponential nature of AI advancement. The plan, a new government website and policy speech called for accelerated AI adoption across federal agencies, and a number of initiatives to make is easier for AI infrastructure expansion, and other measures to ensure American leadership in AI standards. Some leading experts warned that the administration failed to provide sufficient regulations and safeguards for AI safety. Concerns were raised about the negative impacts of cuts to research funding and tightened visa policies for scientists, potentially undermining public trust and America's ability to compete internationally. === September === 7 September The Economist cautioned that AI revenues are relatively modest compared to the high cost and investments in the creation of new data centers. Even Sam Altman, OpenAI CEO and one of the leading figures of the AI boom,, raised concerns about investors' outsized hopes for financial returns. At the same time, history has shown that new technologies, like railways and electricity, endured and spread after the initial hype faded. 12 September Economists warn that U.S. households' direct and indirect investments—mutual funds or retirement plans—in the stock market reached an unprecedented historically high level, now representing 45% of all financial assets, or about $USD51.2 trillion. Compared to the Dot-com bubble this represents a sharp increase in exposure. This makes U.S. households vulnerable to market downturns which in turn would result in decreasing consumer spending. U.S. household net worth rose to a record $176.3 trillion in the second quarter, an increase of $7.3 trillion since early 2025 and about $46 trillion higher than before the pandemic. Federal Reserve data attribute the surge primarily to gains in stock markets and housing values. However, the rise in wealth on paper coincided with increased household borrowing and growing government debt. 18 September Questions were being raised about how quickly the data centers, chips, servers, and GPUs assets of major AI companies will depreciate in value. Comparisons have been made to the Railway Mania in the aftermath of the stock market bubble where a valuable physical infrastructure remained standing, and the telecoms crash after the dot-com bubble which left fiber networks. 28 September There were warnings that record-high American stock ownership during the AI-fueled market boom is a red flag for systemic risk, as the current concentration in equities exceeds levels seen before the dot-com bubble burst in 2000, and could amplify the impact of any future stock market correction. === October === 3 October In 2025 alone, venture capitalists invested almost $USD200 billion in the artificial intelligence sector. 29 October Nvidia was the first company in the world to be valued at US$5 trillion, largely due to AI demand and strategic partnerships with leading technology and AI firms. Nvidia's increase in value was "meteoric". === November === 2 November Forbes reported that, since April, the 'Magnificent Seven' tech giants together contributed over 40% of the S&P 500's return, highlighting their outsized influence and the growing impact of AI on market valuations. CNN warned that while there is a current benefit to investors, with such a high concentration in the S&P 500, they are highly exposed to the fate of the Mag Seven. 2 November Globally there are 11,000 datacentres—huge campuses for AI infrastructure, including thousands of chips, GPUS, and servers. This represents a 500% increase over the last two decades. It is anticipated that $3USDtn more will be spent on increasing that number over the next two or three years. 5 November Concerns about the potential for a market bubble were raised as six of the AI-related Big Tech "Magnificent Seven"—that contribute to the AI boom—reported losing ground in the stock market. Global markets and artificial intelligence have become "deeply intertwined", according to a Reuters report. As of November 2025, more than 50% of the 20 largest S&P firms were deeply exposed to AI. In contrast, in 2000, the 20 S&P 500 firms represented 39% of its total value only 11 of these companies were exposed to the internet. If AI fails to deliver strong returns on their investments, these top S&P firms would be significantly impacted, according to the Economist. Analysts suggest that the AI market in 2025 may not behave like a traditional one, as investors are simultaneously aware of the risks and driven by the potential for outsized rewards. Leading AI labs may believe that the first company to achieve artificial general intelligence (AGI), when an AI system surpasses all human cognitive abilities and becomes capable of self-improvement—could dominate the future of technology and finance. While some have estimated that the potential value of such a breakthrough could be as high as $1.46 quadrillion, this figure is speculative and widely debated. 5 November Bloomberg described Nvidia's H100 Hopper-Blackwell AI chips as the "King of AI chips". Nvidia dominates the AI chip market with over 78% of the market share because of both speed and cost. According to B

Gibberlink

GibberLink is an acoustic data transmission project, with an open-source client available on GitHub, in which two conversational AI agents switch from speaking to one another in a Human-listenable language (such as English) to their own unique language that consists of a sound-level protocol after confirming they are both AI agents. The project was created by Anton Pidkuiko and Boris Starkov. == Reception == The project won the global top prize at the ElevenLabs Worldwide Hackathon. It has also been cited as raising questions around AI ethics and oversight. On February 23, 2025, a YouTube video of two independent conversational ElevenLabs AI agents being prompted to chat about booking a hotel (one as a caller, one as a receptionist) received coverage for going viral. In this video, both agents are prompted to switch to ggwave data-over-sound protocol when they identify the other side as AI, and keep speaking in English otherwise.

Multiple satellite imaging

Multiple satellite imaging is the process of using multiple satellites to gather more information than a single satellite so that a better estimate of the desired source is possible. Something that cannot be resolved with one telescope might be visible with two or more telescopes. == Background == Interferometry is the process of combining waves in such a way that they constructively interfere. When two or more independent sources detect a signal at the same given frequency those signals can be combined and the result is better than each one individually. An overview of Astronomical interferometers and a History of astronomical interferometry can be referenced from their respective pages. The NASA Origins Program was created in the 1990s to ultimately search for the origin of the universe. The theory that the Origins Program is based on is: since light travels at a constant speed until it is absorbed by something; there is still light that was part of the first light ever created traveling about the universe and ultimately some of that light is coming in the general direction of Earth. So a satellite system capable of collecting light from the beginning of the universe would be able to tell us more about where we came from. There is also the constant search for life in other worlds. A satellite system using the interferometric technologies mentioned above would be able to have a much higher resolution than any of the current deep space imaging systems. == Future == NASA is currently focused on the Vision for Space Exploration and has reduced current funding for scientific unmanned space exploration in favor of human exploration. These budget cuts have slowed the multiple satellite imaging development and relevant scientific missions as Project Prometheus and Terrestrial Planet Finder have ended as well but research continues.

Lazy learning

(Not to be confused with the lazy learning regime, see Neural tangent kernel). In machine learning, lazy learning is a learning method in which generalization of the training data is, in theory, delayed until a query is made to the system, as opposed to eager learning, where the system tries to generalize the training data before receiving queries. The primary motivation for employing lazy learning, as in the K-nearest neighbors algorithm, used by online recommendation systems ("people who viewed/purchased/listened to this movie/item/tune also ...") is that the data set is continuously updated with new entries (e.g., new items for sale at Amazon, new movies to view at Netflix, new clips at YouTube, new music at Spotify or Pandora). Because of the continuous update, the "training data" would be rendered obsolete in a relatively short time especially in areas like books and movies, where new best-sellers or hit movies/music are published/released continuously. Therefore, one cannot really talk of a "training phase". Lazy classifiers are most useful for large, continuously changing datasets with few attributes that are commonly queried. Specifically, even if a large set of attributes exist - for example, books have a year of publication, author/s, publisher, title, edition, ISBN, selling price, etc. - recommendation queries rely on far fewer attributes - e.g., purchase or viewing co-occurrence data, and user ratings of items purchased/viewed. == Advantages == The main advantage gained in employing a lazy learning method is that the target function will be approximated locally, such as in the k-nearest neighbor algorithm. Because the target function is approximated locally for each query to the system, lazy learning systems can simultaneously solve multiple problems and deal successfully with changes in the problem domain. At the same time they can reuse a lot of theoretical and applied results from linear regression modelling (notably PRESS statistic) and control. It is said that the advantage of this system is achieved if the predictions using a single training set are only developed for few objects. This can be demonstrated in the case of the k-NN technique, which is instance-based and function is only estimated locally. == Disadvantages == Theoretical disadvantages with lazy learning include: The large space requirement to store the entire training dataset. In practice, this is not an issue because of advances in hardware and the relatively small number of attributes (e.g., as co-occurrence frequency) that need to be stored. Particularly noisy training data increases the case base unnecessarily, because no abstraction is made during the training phase. In practice, as stated earlier, lazy learning is applied to situations where any learning performed in advance soon becomes obsolete because of changes in the data. Also, for the problems for which lazy learning is optimal, "noisy" data does not really occur - the purchaser of a book has either bought another book or hasn't. Lazy learning methods are usually slower to evaluate. In practice, for very large databases with high concurrency loads, the queries are not postponed until actual query time, but recomputed in advance on a periodic basis - e.g., nightly, in anticipation of future queries, and the answers stored. This way, the next time new queries are asked about existing entries in the database, the answers are merely looked up rapidly instead of having to be computed on the fly, which would almost certainly bring a high-concurrency multi-user system to its knees. Larger training data also entail increased cost. Particularly, there is the fixed amount of computational cost, where a processor can only process a limited amount of training data points. There are standard techniques to improve re-computation efficiency so that a particular answer is not recomputed unless the data that impact this answer has changed (e.g., new items, new purchases, new views). In other words, the stored answers are updated incrementally. This approach, used by large e-commerce or media sites, has long been used in the Entrez portal of the National Center for Biotechnology Information (NCBI) to precompute similarities between the different items in its large datasets: biological sequences, 3-D protein structures, published-article abstracts, etc. Because "find similar" queries are asked so frequently, the NCBI uses highly parallel hardware to perform nightly recomputation. The recomputation is performed only for new entries in the datasets against each other and against existing entries: the similarity between two existing entries need not be recomputed. == Examples of Lazy Learning Methods == K-nearest neighbors, which is a special case of instance-based learning. Local regression. Lazy naive Bayes rules, which are extensively used in commercial spam detection software. Here, the spammers keep getting smarter and revising their spamming strategies, and therefore the learning rules must also be continually updated.