EnjunDu/GraphMaster
083
1[
2 {
3 "node_id": 0,
4 "label": 2,
5 "text": "Title: The megaprior heuristic for discovering protein sequence patterns \nAbstract: Several computer algorithms for discovering patterns in groups of protein sequences are in use that are based on fitting the parameters of a statistical model to a group of related sequences. These include hidden Markov model (HMM) algorithms for multiple sequence alignment, and the MEME and Gibbs sampler algorithms for discovering motifs. These algorithms are sometimes prone to producing models that are incorrect because two or more patterns have been combined. The statistical model produced in this situation is a convex combination (weighted average) of two or more different models. This paper presents a solution to the problem of convex combinations in the form of a heuristic based on using extremely low variance Dirichlet mixture priors as part of the statistical model. This heuristic, which we call the megaprior heuristic, increases the strength (i.e., decreases the variance) of the prior in proportion to the size of the sequence dataset. This causes each column in the final model to strongly resemble the mean of a single component of the prior, regardless of the size of the dataset. We describe the cause of the convex combination problem, analyze it mathematically, motivate and describe the implementation of the megaprior heuristic, and show how it can effectively eliminate the problem of convex combinations in protein sequence pattern discovery. ",
6 "neighbors": [
7 8,
8 14,
9 258,
10 435,
11 544
12 ],
13 "mask": "Test"
14 },
15 {
16 "node_id": 1,
17 "label": 5,
18 "text": "Title: Applications of machine learning: a medical follow up study \nAbstract: This paper describes preliminary work that aims to apply some learning strategies to a medical follow-up study. An investigation of the application of three machine learning algorithms-1R, FOIL and InductH to identify risk factors that govern the colposuspension cure rate has been made. The goal of this study is to induce a generalised description or explanation of the classification attribute, colposuspension cure rate (completely cured, improved, unchanged and worse) from the 767 examples in the questionnaires. We looked for a set of rules that described which risk factors result in differences of cure rate. The results were encouraging, and indicate that machine learning can play a useful role in large scale medical problem solving. ",
19 "neighbors": [
20 344
21 ],
22 "mask": "Train"
23 },
24 {
25 "node_id": 2,
26 "label": 4,
27 "text": "Title: Submitted to NIPS96, Section: Applications. Preference: Oral presentation Reinforcement Learning for Dynamic Channel Allocation in\nAbstract: In cellular telephone systems, an important problem is to dynamically allocate the communication resource (channels) so as to maximize service in a stochastic caller environment. This problem is naturally formulated as a dynamic programming problem and we use a reinforcement learning (RL) method to find dynamic channel allocation policies that are better than previous heuristic solutions. The policies obtained perform well for a broad variety of call traffic patterns. We present results on a large cellular system In cellular communication systems, an important problem is to allocate the communication resource (bandwidth) so as to maximize the service provided to a set of mobile callers whose demand for service changes stochastically. A given geographical area is divided into mutually disjoint cells, and each cell serves the calls that are within its boundaries (see Figure 1a). The total system bandwidth is divided into channels, with each channel centered around a frequency. Each channel can be used simultaneously at different cells, provided these cells are sufficiently separated spatially, so that there is no interference between them. The minimum separation distance between simultaneous reuse of the same channel is called the channel reuse constraint . When a call requests service in a given cell either a free channel (one that does not violate the channel reuse constraint) may be assigned to the call, or else the call is blocked from the system; this will happen if no free channel can be found. Also, when a mobile caller crosses from one cell to another, the call is \"handed off\" to the cell of entry; that is, a new free channel is provided to the call at the new cell. If no such channel is available, the call must be dropped/disconnected from the system. One objective of a channel allocation policy is to allocate the available channels to calls so that the number of blocked calls is minimized. An additional objective is to minimize the number of calls that are dropped when they are handed off to a busy cell. These two objectives must be weighted appropriately to reflect their relative importance, since dropping existing calls is generally more undesirable than blocking new calls. with approximately 70 49 states.",
28 "neighbors": [
29 410,
30 471,
31 552,
32 565
33 ],
34 "mask": "Train"
35 },
36 {
37 "node_id": 3,
38 "label": 4,
39 "text": "Title: Planning and Acting in Partially Observable Stochastic Domains \nAbstract: In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable mdps (pomdps). We then outline a novel algorithm for solving pomdps off line and show how, in some cases, a finite-memory controller can be extracted from the solution to a pomdp. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions. Consider the problem of a robot navigating in a large office building. The robot can move from hallway intersection to intersection and can make local observations of its world. Its actions are not completely reliable, however. Sometimes, when it intends to move, it stays where it is or goes too far; sometimes, when it intends to turn, it overshoots. It has similar problems with observation. Sometimes a corridor looks like a corner; sometimes a T-junction looks like an L-junction. How can such an error-plagued robot navigate, even given a map of the corridors? In general, the robot will have to remember something about its history of actions and observations and use this information, together with its knowledge of the underlying dynamics of the world (the map and other information), to maintain an estimate of its location. Many engineering applications follow this approach, using methods like the Kalman filter [18] to maintain a running estimate of the robot's spatial uncertainty, expressed as an ellipsoid or normal distribution in Cartesian space. This approach will not do for our robot, though. Its uncertainty may be discrete: it might be almost certain that it is in the north-east corner of either the fourth or the seventh floors, though it admits a chance that it is on the fifth floor, as well. Then, given an uncertain estimate of its location, the robot has to decide what actions to take. In some cases, it might be sufficient to ignore its uncertainty and take actions that would be appropriate for the most likely location. In other cases, it might be better for ",
40 "neighbors": [
41 197,
42 463,
43 601
44 ],
45 "mask": "Test"
46 },
47 {
48 "node_id": 4,
49 "label": 3,
50 "text": "Note: c Massachusetts Institute of Technology The thesis consists of the development of this Michael I. Jordan Title: Professor \nAbstract: Graphical models enhance the representational power of probability models through qualitative characterization of their properties. This also leads to greater efficiency in terms of the computational algorithms that empower such representations. The increasing complexity of these models, however, quickly renders exact probabilistic calculations infeasible. We propose a principled framework for approximating graphical models based on variational methods. We develop variational techniques from the perspective that unifies and expands their applicability to graphical models. These methods allow the (recursive) computation of upper and lower bounds on the quantities of interest. Such bounds yield considerably more information than mere approximations and provide an inherent error metric for tailoring the approximations individually to the cases considered. These desirable properties, concomitant to the variational methods, are unlikely to arise as a result of other deterministic or stochastic approximations. ",
51 "neighbors": [
52 170
53 ],
54 "mask": "Train"
55 },
56 {
57 "node_id": 5,
58 "label": 3,
59 "text": "Title: Some Experiments with Real-time Decision Algorithms \nAbstract: Real-time Decision algorithms are a class of incremental resource-bounded [Horvitz, 89] or anytime [Dean, 93] algorithms for evaluating influence diagrams. We present a test domain for real-time decision algorithms, and the results of experiments with several Real-time Decision Algorithms in this domain. The results demonstrate high performance for two algorithms, a decision-evaluation variant of Incremental Probabilisitic Inference [DAmbrosio, 93] and a variant of an algorithm suggested by Goldszmidt, [Goldszmidt, 95], PK-reduced. We discuss the implications of these experimental results and explore the broader applicability of these algorithms.",
60 "neighbors": [
61 490,
62 2164
63 ],
64 "mask": "Train"
65 },
66 {
67 "node_id": 6,
68 "label": 6,
69 "text": "Title: A Formal Framework for Speedup Learning from Problems and Solutions \nAbstract: Speedup learning seeks to improve the computational efficiency of problem solving with experience. In this paper, we develop a formal framework for learning efficient problem solving from random problems and their solutions. We apply this framework to two different representations of learned knowledge, namely control rules and macro-operators, and prove theorems that identify sufficient conditions for learning in each representation. Our proofs are constructive in that they are accompanied with learning algorithms. Our framework captures both empirical and explanation-based speedup learning in a unified fashion. We illustrate our framework with implementations in two domains: symbolic integration and Eight Puzzle. This work integrates many strands of experimental and theoretical work in machine learning, including empirical learning of control rules, macro-operator learning, ",
70 "neighbors": [
71 251,
72 490
73 ],
74 "mask": "Train"
75 },
76 {
77 "node_id": 7,
78 "label": 2,
79 "text": "Title: Optimal Alignments in Linear Space using Automaton-derived Cost Functions (Extended Abstract) Submitted to CPM'96 \nAbstract: In a previous paper [SM95], we showed how finite automata could be used to define objective functions for assessing the quality of an alignment of two (or more) sequences. In this paper, we show some results of using such cost functions. We also show how to extend Hischberg's linear space algorithm [Hir75] to this setting, thus generalizing a result of Myers and Miller [MM88b]. ",
80 "neighbors": [
81 258
82 ],
83 "mask": "Test"
84 },
85 {
86 "node_id": 8,
87 "label": 2,
88 "text": "Title: Meta-MEME: Motif-based Hidden Markov Models of Protein Families \nAbstract: In a previous paper [SM95], we showed how finite automata could be used to define objective functions for assessing the quality of an alignment of two (or more) sequences. In this paper, we show some results of using such cost functions. We also show how to extend Hischberg's linear space algorithm [Hir75] to this setting, thus generalizing a result of Myers and Miller [MM88b]. ",
89 "neighbors": [
90 0,
91 14,
92 258,
93 435,
94 751
95 ],
96 "mask": "Validation"
97 },
98 {
99 "node_id": 9,
100 "label": 6,
101 "text": "Title: Online Learning versus O*ine Learning \nAbstract: We present an off-line variant of the mistake-bound model of learning. Just like in the well studied on-line model, a learner in the offline model has to learn an unknown concept from a sequence of elements of the instance space on which he makes \"guess and test\" trials. In both models, the aim of the learner is to make as few mistakes as possible. The difference between the models is that, while in the on-line model only the set of possible elements is known, in the off-line model the sequence of elements (i.e., the identity of the elements as well as the order in which they are to be presented) is known to the learner in advance. We give a combinatorial characterization of the number of mistakes in the off-line model. We apply this characterization to solve several natural questions that arise for the new model. First, we compare the mistake bounds of an off-line learner to those of a learner learning the same concept classes in the on-line scenario. We show that the number of mistakes in the on-line learning is at most a log n factor more than the off-line learning, where n is the length of the sequence. In addition, we show that if there is an off-line algorithm that does not make more than a constant number of mistakes for each sequence then there is an online algorithm that also does not make more than a constant number of mistakes. The second issue we address is the effect of the ordering of the elements on the number of mistakes of an off-line learner. It turns out that there are sequences on which an off-line learner can guarantee at most one mistake, yet a permutation of the same sequence forces him to err on many elements. We prove, however, that the gap, between the off-line mistake bounds on permutations of the same sequence of n-many elements, cannot be larger than a multiplicative factor of log n, and we present examples that obtain such a gap. ",
102 "neighbors": [
103 308,
104 453,
105 481,
106 761
107 ],
108 "mask": "Train"
109 },
110 {
111 "node_id": 10,
112 "label": 2,
113 "text": "Title: GRKPACK: FITTING SMOOTHING SPLINE ANOVA MODELS FOR EXPONENTIAL FAMILIES \nAbstract: Wahba, Wang, Gu, Klein and Klein (1995) introduced Smoothing Spline ANalysis of VAriance (SS ANOVA) method for data from exponential families. Based on RKPACK, which fits SS ANOVA models to Gaussian data, we introduce GRKPACK: a collection of Fortran subroutines for binary, binomial, Poisson and Gamma data. We also show how to calculate Bayesian confidence intervals for SS ANOVA estimates. ",
114 "neighbors": [
115 192,
116 193,
117 280,
118 420,
119 519
120 ],
121 "mask": "Train"
122 },
123 {
124 "node_id": 11,
125 "label": 1,
126 "text": "Title: Simple Genetic Programming for Supervised Learning Problems \nAbstract: This paper presents an evolutionary approach to finding learning rules to several supervised tasks. In this approach potential solutions are represented as variable length mathematical LISP S-expressions. Thus, it is similar to Genetic Programming (GP) but it employs a fixed set of non-problem-specific functions to solve a variety of problems. In this paper three Monk's and parity problems are tested. The results indicate the usefulness of the encoding schema in discovering learning rules for supervised learning problems with the emphasis on hard learning problems. The problems and future research directions are discussed within the context of GP practices. ",
127 "neighbors": [
128 624,
129 659
130 ],
131 "mask": "Train"
132 },
133 {
134 "node_id": 12,
135 "label": 3,
136 "text": "Title: Estimating Bayes Factors via Posterior Simulation with the Laplace-Metropolis Estimator \nAbstract: The key quantity needed for Bayesian hypothesis testing and model selection is the marginal likelihood for a model, also known as the integrated likelihood, or the marginal probability of the data. In this paper we describe a way to use posterior simulation output to estimate marginal likelihoods. We describe the basic Laplace-Metropolis estimator for models without random effects. For models with random effects the compound Laplace-Metropolis estimator is introduced. This estimator is applied to data from the World Fertility Survey and shown to give accurate results. Batching of simulation output is used to assess the uncertainty involved in using the compound Laplace-Metropolis estimator. The method allows us to test for the effects of independent variables in a random effects model, and also to test for the presence of the random effects.",
137 "neighbors": [
138 84,
139 155
140 ],
141 "mask": "Validation"
142 },
143 {
144 "node_id": 13,
145 "label": 0,
146 "text": "Title: Unifying Empirical and Explanation-Based Learning by Modeling the Utility of Learned Knowledge \nAbstract: The overfit problem in empirical learning and the utility problem in explanation-based learning describe a similar phenomenon: the degradation of performance due to an increase in the amount of learned knowledge. Plotting the performance of learned knowledge during the course of learning (the performance response) reveals a common trend for several learning methods. Modeling this trend allows a control system to constrain the amount of learned knowledge to achieve peak performance and avoid the general utility problem. Experiments evaluate a particular empirical model of the trend, and analysis of the learners derive several formal models. If, as evidence suggests, the general utility problem can be modeled using the same mechanisms for different learning paradigms, then the model serves to unify the paradigms into one framework capable of comparing and selecting different learning methods based on predicted achievable performance.",
147 "neighbors": [
148 482,
149 578,
150 1234
151 ],
152 "mask": "Train"
153 },
154 {
155 "node_id": 14,
156 "label": 2,
157 "text": "Title: Hidden Markov Models in Computational Biology: Applications to Protein Modeling UCSC-CRL-93-32 Keywords: Hidden Markov Models,\nAbstract: Hidden Markov Models (HMMs) are applied to the problems of statistical modeling, database searching and multiple sequence alignment of protein families and protein domains. These methods are demonstrated on the globin family, the protein kinase catalytic domain, and the EF-hand calcium binding motif. In each case the parameters of an HMM are estimated from a training set of unaligned sequences. After the HMM is built, it is used to obtain a multiple alignment of all the training sequences. It is also used to search the SWISS-PROT 22 database for other sequences that are members of the given protein family, or contain the given domain. The HMM produces multiple alignments of good quality that agree closely with the alignments produced by programs that incorporate three-dimensional structural information. When employed in discrimination tests (by examining how closely the sequences in a database fit the globin, kinase and EF-hand HMMs), the HMM is able to distinguish members of these families from non-members with a high degree of accuracy. Both the HMM and PRO-FILESEARCH (a technique used to search for relationships between a protein sequence and multiply aligned sequences) perform better in these tests than PROSITE (a dictionary of sites and patterns in proteins). The HMM appears to have a slight advantage ",
158 "neighbors": [
159 0,
160 8,
161 31,
162 232,
163 242,
164 258,
165 268,
166 384,
167 393,
168 400,
169 435,
170 437,
171 443,
172 544,
173 613,
174 708,
175 736,
176 746,
177 751
178 ],
179 "mask": "Train"
180 },
181 {
182 "node_id": 15,
183 "label": 2,
184 "text": "Title: Back Propagation is Sensitive to Initial Conditions \nAbstract: This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the back-propagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of back propagation to initial weight configuration. ",
185 "neighbors": [
186 80,
187 129,
188 152,
189 234,
190 253,
191 254,
192 322,
193 399,
194 538,
195 701
196 ],
197 "mask": "Train"
198 },
199 {
200 "node_id": 16,
201 "label": 4,
202 "text": "Title: Exploration in Active Learning \nAbstract: This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the back-propagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of back propagation to initial weight configuration. ",
203 "neighbors": [
204 466,
205 552,
206 566,
207 1697
208 ],
209 "mask": "Train"
210 },
211 {
212 "node_id": 17,
213 "label": 2,
214 "text": "Title: A Neural Network Model of Memory Consolidation \nAbstract: Some forms of memory rely temporarily on a system of brain structures located in the medial temporal lobe that includes the hippocampus. The recall of recent events is one task that relies crucially on the proper functioning of this system. As the event becomes less recent, the medial temporal lobe becomes less critical to the recall of the event, and the recollection appears to rely more upon the neocortex. It has been proposed that a process called consolidation is responsible for transfer of memory from the medial temporal lobe to the neocortex. We examine a network model proposed by P. Alvarez and L. Squire designed to incorporate some of the known features of consolidation, and propose several possible experiments intended to help evaluate the performance of this model under more realistic conditions. Finally, we implement an extended version of the model that can accommodate varying assumptions about the number of areas and connections within the brain and memory capacity, and examine the performance of our model on Alvarez and Squire's original task. ",
215 "neighbors": [
216 146
217 ],
218 "mask": "Train"
219 },
220 {
221 "node_id": 18,
222 "label": 2,
223 "text": "Title: Topography And Ocular Dominance: A Model Exploring Positive Correlations \nAbstract: The map from eye to brain in vertebrates is topographic, i.e. neighbouring points in the eye map to neighbouring points in the brain. In addition, when two eyes innervate the same target structure, the two sets of fibres segregate to form ocular dominance stripes. Experimental evidence from the frog and goldfish suggests that these two phenomena may be subserved by the same mechanisms. We present a computational model that addresses the formation of both topography and ocular dominance. The model is based on a form of competitive learning with subtractive enforcement of a weight normalization rule. Inputs to the model are distributed patterns of activity presented simultaneously in both eyes. An important aspect of this model is that ocular dominance segregation can occur when the two eyes are positively correlated, whereas previous models have tended to assume zero or negative correlations between the eyes. This allows investigation of the dependence of the pattern of stripes on the degree of correlation between the eyes: we find that increasing correlation leads to narrower stripes. Experiments are suggested to test this prediction.",
224 "neighbors": [
225 127,
226 427,
227 745,
228 747,
229 866,
230 890,
231 1932
232 ],
233 "mask": "Train"
234 },
235 {
236 "node_id": 19,
237 "label": 2,
238 "text": "Title: Validation of Average Error Rate Over Classifiers \nAbstract: We examine methods to estimate the average and variance of test error rates over a set of classifiers. We begin with the process of drawing a classifier at random for each example. Given validation data, the average test error rate can be estimated as if validating a single classifier. Given the test example inputs, the variance can be computed exactly. Next, we consider the process of drawing a classifier at random and using it on all examples. Once again, the expected test error rate can be validated as if validating a single classifier. However, the variance must be estimated by validating all classifers, which yields loose or uncertain bounds. ",
239 "neighbors": [
240 74,
241 571
242 ],
243 "mask": "Train"
244 },
245 {
246 "node_id": 20,
247 "label": 6,
248 "text": "Title: 25 Learning in Hybrid Noise Environments Using Statistical Queries \nAbstract: We consider formal models of learning from noisy data. Specifically, we focus on learning in the probability approximately correct model as defined by Valiant. Two of the most widely studied models of noise in this setting have been classification noise and malicious errors. However, a more realistic model combining the two types of noise has not been formalized. We define a learning environment based on a natural combination of these two noise models. We first show that hypothesis testing is possible in this model. We next describe a simple technique for learning in this model, and then describe a more powerful technique based on statistical query learning. We show that the noise tolerance of this improved technique is roughly optimal with respect to the desired learning accuracy and that it provides a smooth tradeoff between the tolerable amounts of the two types of noise. Finally, we show that statistical query simulation yields learning algorithms for other combinations of noise models, thus demonstrating that statistical query specification truly An important goal of research in machine learning is to determine which tasks can be automated, and for those which can, to determine their information and computation requirements. One way to answer these questions is through the development and investigation of formal models of machine learning which capture the task of learning under plausible assumptions. In this work, we consider the formal model of learning from examples called \"probably approximately correct\" (PAC) learning as defined by Valiant [Val84]. In this setting, a learner attempts to approximate an unknown target concept simply by viewing positive and negative examples of the concept. An adversary chooses, from some specified function class, a hidden f0; 1g-valued target function defined over some specified domain of examples and chooses a probability distribution over this domain. The goal of the learner is to output in both polynomial time and with high probability, an hypothesis which is \"close\" to the target function with respect to the distribution of examples. The learner gains information about the target function and distribution by interacting with an example oracle. At each request by the learner, this oracle draws an example randomly according to the hidden distribution, labels it according to the hidden target function, and returns the labelled example to the learner. A class of functions F is said to be PAC learnable if captures the generic fault tolerance of a learning algorithm.",
249 "neighbors": [
250 25,
251 267,
252 334,
253 640,
254 732
255 ],
256 "mask": "Train"
257 },
258 {
259 "node_id": 21,
260 "label": 4,
261 "text": "Title: Decision Tree Function Approximation in Reinforcement Learning \nAbstract: We present a decision tree based approach to function approximation in reinforcement learning. We compare our approach with table lookup and a neural network function approximator on three problems: the well known mountain car and pole balance problems as well as a simulated automobile race car. We find that the decision tree can provide better learning performance than the neural network function approximation and can solve large problems that are infeasible using table lookup.",
262 "neighbors": [
263 294,
264 438,
265 567,
266 1378
267 ],
268 "mask": "Test"
269 },
270 {
271 "node_id": 22,
272 "label": 1,
273 "text": "Title: Discovering Complex Othello Strategies Through Evolutionary Neural Networks \nAbstract: An approach to develop new game playing strategies based on artificial evolution of neural networks is presented. Evolution was directed to discover strategies in Othello against a random-moving opponent and later against an ff-fi search program. The networks discovered first a standard positional strategy, and subsequently a mobility strategy, an advanced strategy rarely seen outside of tournaments. The latter discovery demonstrates how evolutionary neural networks can develop novel solutions by turning an initial disadvantage into an advantage in a changed environment. ",
274 "neighbors": [
275 129,
276 163,
277 191,
278 1790,
279 2257
280 ],
281 "mask": "Train"
282 },
283 {
284 "node_id": 23,
285 "label": 3,
286 "text": "Title: Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses \nAbstract: Technical Report No. 670 December, 1997 ",
287 "neighbors": [
288 41,
289 759
290 ],
291 "mask": "Train"
292 },
293 {
294 "node_id": 24,
295 "label": 4,
296 "text": "Title: The Role of Transfer in Learning (extended abstract) \nAbstract: Technical Report No. 670 December, 1997 ",
297 "neighbors": [
298 39,
299 269
300 ],
301 "mask": "Test"
302 },
303 {
304 "node_id": 25,
305 "label": 6,
306 "text": "Title: General Bounds on Statistical Query Learning and PAC Learning with Noise via Hypothesis Boosting \nAbstract: We derive general bounds on the complexity of learning in the Statistical Query model and in the PAC model with classification noise. We do so by considering the problem of boosting the accuracy of weak learning algorithms which fall within the Statistical Query model. This new model was introduced by Kearns [12] to provide a general framework for efficient PAC learning in the presence of classification noise. We first show a general scheme for boosting the accuracy of weak SQ learning algorithms, proving that weak SQ learning is equivalent to strong SQ learning. The boosting is efficient and is used to show our main result of the first general upper bounds on the complexity of strong SQ learning. Specifically, we derive simultaneous upper bounds with respect to * on the number of queries, O(log 2 1 * ), the Vapnik-Chervonenkis dimension of the query space, O(log 1 * ), and the inverse of the minimum tolerance, O( 1 * log 1 * ). In addition, we show that these general upper bounds are nearly optimal by describing a class of learning problems for which we simultaneously lower bound the number of queries by (log 1 * ) We further apply our boosting results in the SQ model to learning in the PAC model with classification noise. Since nearly all PAC learning algorithms can be cast in the SQ model, we can apply our boosting techniques to convert these PAC algorithms into highly efficient SQ algorithms. By simulating these efficient SQ algorithms in the PAC model with classification noise, we show that nearly all PAC algorithms can be converted into highly efficient PAC algorithms which tolerate classification noise. We give an upper bound on the sample complexity of these noise-tolerant PAC algorithms which is nearly optimal with respect to the noise rate. We also give upper bounds on space complexity and hypothesis size and show that these two measures are in fact independent of the noise rate. We note that the running times of these noise-tolerant PAC algorithms are efficient. This sequence of simulations also demonstrates that it is possible to boost the accuracy of nearly all PAC algorithms even in the presence of noise. This provides a partial answer to an open problem of Schapire [15] and the first theoretical evidence for an empirical result of Drucker, Schapire and Simard [4]. ",
307 "neighbors": [
308 20,
309 456,
310 778,
311 1181,
312 1897
313 ],
314 "mask": "Test"
315 },
316 {
317 "node_id": 26,
318 "label": 2,
319 "text": "Title: Neural Network Applicability: Classifying the Problem Space \nAbstract: The tremendous current effort to propose neurally inspired methods of computation forces closer scrutiny of real world application potential of these models. This paper categorizes applications into classes and particularly discusses features of applications which make them efficiently amenable to neural network methods. Computational machines do deterministic mappings of inputs to outputs and many computational mechanisms have been proposed for problem solutions. Neural network features include parallel execution, adaptive learning, generalization, and fault tolerance. Often, much effort is given to a model and applications which can already be implemented in a much more efficient way with an alternate technology. Neural networks are potentially powerful devices for many classes of applications, but not all. However, it is proposed that the class of applications for which neural networks are efficient is both large and commonly occurring in nature. Comparison of supervised, unsupervised, and generalizing systems is also included. ",
320 "neighbors": [
321 747,
322 1129,
323 2612
324 ],
325 "mask": "Test"
326 },
327 {
328 "node_id": 27,
329 "label": 3,
330 "text": "Title: Formal Rules for Selecting Prior Distributions: A Review and Annotated Bibliography \nAbstract: Subjectivism has become the dominant philosophical foundation for Bayesian inference. Yet, in practice, most Bayesian analyses are performed with so-called \"noninfor-mative\" priors, that is, priors constructed by some formal rule. We review the plethora of techniques for constructing such priors, and discuss some of the practical and philosophical issues that arise when they are used. We give special emphasis to Jeffreys's rules and discuss the evolution of his point of view about the interpretation of priors, away from unique representation of ignorance toward the notion that they should be chosen by convention. We conclude that the problems raised by the research on priors chosen by formal rules are serious and may not be dismissed lightly; when sample sizes are small (relative to the number of parameters being estimated) it is dangerous to put faith in any \"default\" solution; but when asymptotics take over, Jeffreys's rules and their variants remain reasonable choices. We also provide an annotated bibliography. fl Robert E. Kass is Professor and Larry Wasserman is Associate Professor, Department of Statistics, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213-2717. The work of both authors was supported by NSF grant DMS-9005858 and NIH grant R01-CA54852-01. The authors thank Nick Polson for helping with a few annotations, and Jim Berger, Teddy Seidenfeld and Arnold Zellner for useful comments and discussion. ",
331 "neighbors": [
332 84,
333 532
334 ],
335 "mask": "Train"
336 },
337 {
338 "node_id": 28,
339 "label": 2,
340 "text": "Title: A Delay Damage Model Selection Algorithm for NARX Neural Networks \nAbstract: Recurrent neural networks have become popular models for system identification and time series prediction. NARX (Nonlinear AutoRegressive models with eXogenous inputs) neural network models are a popular subclass of recurrent networks and have been used in many applications. Though embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction. ",
341 "neighbors": [
342 586,
343 611,
344 1606,
345 1718
346 ],
347 "mask": "Validation"
348 },
349 {
350 "node_id": 29,
351 "label": 5,
352 "text": "Title: Stochastically Guided Disjunctive Version Space Learning \nAbstract: This paper presents an incremental concept learning approach to identiflcation of concepts with high overall accuracy. The main idea is to address the concept overlap as a central problem when learning multiple descriptions. Many traditional inductive algorithms, as those from the disjunctive version space family considered here, face this problem. The approach focuses on combinations of confldent, possibly overlapping, concepts with an original stochastic complexity formula. The focusing is e-cient because it is organized as a simulated annealing-based beam search. The experiments show that the approach is especially suitable for developing incremental learning algorithms with the following advantages: flrst, it generates highly accurate concepts; second, it overcomes to a certain degree the sensitivity to the order of examples; and third, it handles noisy examples. ",
353 "neighbors": [
354 319,
355 382,
356 426,
357 429
358 ],
359 "mask": "Train"
360 },
361 {
362 "node_id": 30,
363 "label": 0,
364 "text": "Title: Towards More Creative Case-Based Design Systems \nAbstract: Case-based reasoning (CBR) has a great deal to offer in supporting creative design, particularly processes that rely heavily on previous design experience, such as framing the problem and evaluating design alternatives. However, most existing CBR systems are not living up to their potential. They tend to adapt and reuse old solutions in routine ways, producing robust but uninspired results. Little research effort has been directed towards the kinds of situation assessment, evaluation, and assimilation processes that facilitate the exploration of ideas and the elaboration and redefinition of problems that are crucial to creative design. Also, their typically rigid control structures do not facilitate the kinds of strategic control and opportunism inherent in creative reasoning. In this paper, we describe the types of behavior we would like case-based design systems to support, based on a study of designers working on a mechanical engineering problem. We show how the standard CBR framework should be extended and we describe an architecture we are developing to experiment with these ideas. 1 ",
365 "neighbors": [
366 231,
367 285,
368 679,
369 1148,
370 2276
371 ],
372 "mask": "Train"
373 },
374 {
375 "node_id": 31,
376 "label": 2,
377 "text": "Title: GIBBS-MARKOV MODELS \nAbstract: In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. ",
378 "neighbors": [
379 14,
380 250,
381 1116
382 ],
383 "mask": "Train"
384 },
385 {
386 "node_id": 32,
387 "label": 0,
388 "text": "Title: Design by Interactive Exploration Using Memory-Based Techniques \nAbstract: One of the characteristics of design is that designers rely extensively on past experience in order to create new designs. Because of this, memory-based techniques from artificial intelligence, which help store, organise, retrieve, and reuse experiential knowledge held in memory, are good candidates for aiding designers. Another characteristic of design is the phenomenon of exploration in the early stages of design configuration. A designer begins with an ill-structured, partially defined, problem specification, and through a process of exploration gradually refines and modifies it as his/her understanding of the problem improves. In this paper we describe demex, an interactive computer-aided design system that employs memory-based techniques to help its users explore the design problems they pose to the system, in order to acquire a better understanding of the requirements of the problems. demex has been applied in the domain of structural design of buildings. ",
389 "neighbors": [
390 679
391 ],
392 "mask": "Validation"
393 },
394 {
395 "node_id": 33,
396 "label": 2,
397 "text": "Title: Learning Generative Models with the Up-Propagation Algorithm \nAbstract: Up-propagation is an algorithm for inverting and learning neural network generative models. Sensory input is processed by inverting a model that generates patterns from hidden variables using top-down connections. The inversion process is iterative, utilizing a negative feedback loop that depends on an error signal propagated by bottom-up connections. The error signal is also used to learn the generative model from examples. The algorithm is benchmarked against principal component analysis in In his doctrine of unconscious inference, Helmholtz argued that perceptions are formed by the interaction of bottom-up sensory data with top-down expectations. According to one interpretation of this doctrine, perception is a procedure of sequential hypothesis testing. We propose a new algorithm, called up-propagation, that realizes this interpretation in layered neural networks. It uses top-down connections to generate hypotheses, and bottom-up connections to revise them. It is important to understand the difference between up-propagation and its ancestor, the backpropagation algorithm[1]. Backpropagation is a learning algorithm for recognition models. As shown in Figure 1a, bottom-up connections recognize patterns, while top-down connections propagate an error signal that is used to learn the recognition model. In contrast, up-propagation is an algorithm for inverting and learning generative models, as shown in Figure 1b. Top-down connections generate patterns from a set of hidden variables. Sensory input is processed by inverting the generative model, recovering hidden variables that could have generated the sensory data. This operation is called either pattern recognition or pattern analysis, depending on the meaning of the hidden variables. Inversion of the generative model is done iteratively, through a negative feedback loop driven by an error signal from the bottom-up connections. The error signal is also used for learning the connections experiments on images of handwritten digits.",
398 "neighbors": [
399 250,
400 1591,
401 1701
402 ],
403 "mask": "Train"
404 },
405 {
406 "node_id": 34,
407 "label": 4,
408 "text": "Title: Using a Case Base of Surfaces to Speed-Up Reinforcement Learning \nAbstract: This paper demonstrates the exploitation of certain vision processing techniques to index into a case base of surfaces. The surfaces are the result of reinforcement learning and represent the optimum choice of actions to achieve some goal from anywhere in the state space. This paper shows how strong features that occur in the interaction of the system with its environment can be detected early in the learning process. Such features allow the system to identify when an identical, or very similar, task has been solved previously and to retrieve the relevant surface. This results in an orders of magnitude increase in learning rate. ",
409 "neighbors": [
410 35,
411 66,
412 559,
413 565,
414 566
415 ],
416 "mask": "Train"
417 },
418 {
419 "node_id": 35,
420 "label": 4,
421 "text": "Title: A Teaching Strategy for Memory-Based Control \nAbstract: Combining different machine learning algorithms in the same system can produce benefits above and beyond what either method could achieve alone. This paper demonstrates that genetic algorithms can be used in conjunction with lazy learning to solve examples of a difficult class of delayed reinforcement learning problems better than either method alone. This class, the class of differential games, includes numerous important control problems that arise in robotics, planning, game playing, and other areas, and solutions for differential games suggest solution strategies for the general class of planning and control problems. We conducted a series of experiments applying three learning approaches|lazy Q-learning, k-nearest neighbor (k-NN), and a genetic algorithm|to a particular differential game called a pursuit game. Our experiments demonstrate that k-NN had great difficulty solving the problem, while a lazy version of Q-learning performed moderately well and the genetic algorithm performed even better. These results motivated the next step in the experiments, where we hypothesized k-NN was having difficulty because it did not have good examples-a common source of difficulty for lazy learning. Therefore, we used the genetic algorithm as a bootstrapping method for k-NN to create a system to provide these examples. Our experiments demonstrate that the resulting joint system learned to solve the pursuit games with a high degree of accuracy-outperforming either method alone-and with relatively small memory requirements.",
422 "neighbors": [
423 34
424 ],
425 "mask": "Train"
426 },
427 {
428 "node_id": 36,
429 "label": 2,
430 "text": "Title: Generative Models for Discovering Sparse Distributed Representations \nAbstract: We describe a hierarchical, generative model that can be viewed as a non-linear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demon strate that the network learns to extract sparse, distributed, hierarchical representations.",
431 "neighbors": [
432 257,
433 745,
434 1066,
435 1591,
436 1701,
437 1933,
438 1974,
439 2072,
440 2390
441 ],
442 "mask": "Train"
443 },
444 {
445 "node_id": 37,
446 "label": 4,
447 "text": "Title: Hierarchical Evolution of Neural Networks \nAbstract: In most applications of neuro-evolution, each individual in the population represents a complete neural network. Recent work on the SANE system, however, has demonstrated that evolving individual neurons often produces a more efficient genetic search. This paper demonstrates that while SANE can solve easy tasks very quickly, it often stalls in larger problems. A hierarchical approach to neuro-evolution is presented that overcomes SANE's difficulties by integrating both a neuron-level exploratory search and a network-level exploitive search. In a robot arm manipulation task, the hierarchical approach outperforms both a neuron-based search and a network-based search. ",
448 "neighbors": [
449 563
450 ],
451 "mask": "Train"
452 },
453 {
454 "node_id": 38,
455 "label": 1,
456 "text": "Title: HOW TO EVOLVE AUTONOMOUS ROBOTS: DIFFERENT APPROACHES IN EVOLUTIONARY ROBOTICS \nAbstract: In most applications of neuro-evolution, each individual in the population represents a complete neural network. Recent work on the SANE system, however, has demonstrated that evolving individual neurons often produces a more efficient genetic search. This paper demonstrates that while SANE can solve easy tasks very quickly, it often stalls in larger problems. A hierarchical approach to neuro-evolution is presented that overcomes SANE's difficulties by integrating both a neuron-level exploratory search and a network-level exploitive search. In a robot arm manipulation task, the hierarchical approach outperforms both a neuron-based search and a network-based search. ",
457 "neighbors": [
458 219,
459 273,
460 372,
461 563,
462 1689
463 ],
464 "mask": "Validation"
465 },
466 {
467 "node_id": 39,
468 "label": 4,
469 "text": "Title: Finding Structure in Reinforcement Learning \nAbstract: Reinforcement learning addresses the problem of learning to select actions in order to maximize one's performance in unknown environments. To scale reinforcement learning to complex real-world tasks, such as typically studied in AI, one must ultimately be able to discover the structure in the world, in order to abstract away the myriad of details and to operate in more tractable problem spaces. This paper presents the SKILLS algorithm. SKILLS discovers skills, which are partially defined action policies that arise in the context of multiple, related tasks. Skills collapse whole action sequences into single operators. They are learned by minimizing the compactness of action policies, using a description length argument on their representation. Empirical results in simple grid navigation tasks illustrate the successful discovery of structure in reinforcement learning.",
470 "neighbors": [
471 24,
472 82
473 ],
474 "mask": "Train"
475 },
476 {
477 "node_id": 40,
478 "label": 6,
479 "text": "Title: On-line Learning with Linear Loss Constraints \nAbstract: We consider a generalization of the mistake-bound model (for learning f0; 1g-valued functions) in which the learner must satisfy a general constraint on the number M + of incorrect 1 predictions and the number M of incorrect 0 predictions. We describe a general-purpose optimal algorithm for our formulation of this problem. We describe several applications of our general results, involving situations in which the learner wishes to satisfy linear inequalities in M + and M .",
480 "neighbors": [
481 761
482 ],
483 "mask": "Train"
484 },
485 {
486 "node_id": 41,
487 "label": 3,
488 "text": "Title: Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review \nAbstract: A critical issue for users of Markov Chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but currently has yielded relatively little that is of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of thirteen convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all the methods can fail to detect the sorts of convergence failure they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including applying diagnostic procedures to a small number of parallel chains, monitoring autocorrelations and cross-correlations, and modifying parameterizations or sampling algorithms appropriately. We emphasize, however, that it is not possible to say with certainty that a finite sample from an MCMC algorithm is representative of an underlying stationary distribution. Mary Kathryn Cowles is Assistant Professor of Biostatistics, Harvard School of Public Health, Boston, MA 02115. Bradley P. Carlin is Associate Professor, Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455. Much of the work was done while the first author was a graduate student in the Divison of Biostatistics at the University of Minnesota and then Assistant Professor, Biostatistics Section, Department of Preventive and Societal Medicine, University of Nebraska Medical Center, Omaha, NE 68198. The work of both authors was supported in part by National Institute of Allergy and Infectious Diseases FIRST Award 1-R29-AI33466. The authors thank the developers of the diagnostics studied here for sharing their insights, experiences, and software, and Drs. Thomas Louis and Luke Tierney for helpful discussions and suggestions which greatly improved the manuscript. ",
489 "neighbors": [
490 23,
491 94,
492 115,
493 352,
494 533,
495 725,
496 888,
497 889,
498 892,
499 1713,
500 1716,
501 1733,
502 1982,
503 2456
504 ],
505 "mask": "Train"
506 },
507 {
508 "node_id": 42,
509 "label": 1,
510 "text": "Title: Evolutionary Module Acquisition \nAbstract: A critical issue for users of Markov Chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but currently has yielded relatively little that is of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of thirteen convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all the methods can fail to detect the sorts of convergence failure they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including applying diagnostic procedures to a small number of parallel chains, monitoring autocorrelations and cross-correlations, and modifying parameterizations or sampling algorithms appropriately. We emphasize, however, that it is not possible to say with certainty that a finite sample from an MCMC algorithm is representative of an underlying stationary distribution. Mary Kathryn Cowles is Assistant Professor of Biostatistics, Harvard School of Public Health, Boston, MA 02115. Bradley P. Carlin is Associate Professor, Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455. Much of the work was done while the first author was a graduate student in the Divison of Biostatistics at the University of Minnesota and then Assistant Professor, Biostatistics Section, Department of Preventive and Societal Medicine, University of Nebraska Medical Center, Omaha, NE 68198. The work of both authors was supported in part by National Institute of Allergy and Infectious Diseases FIRST Award 1-R29-AI33466. The authors thank the developers of the diagnostics studied here for sharing their insights, experiences, and software, and Drs. Thomas Louis and Luke Tierney for helpful discussions and suggestions which greatly improved the manuscript. ",
511 "neighbors": [
512 163,
513 188,
514 189,
515 793
516 ],
517 "mask": "Train"
518 },
519 {
520 "node_id": 43,
521 "label": 2,
522 "text": "Title: Competitive Anti-Hebbian Learning of Invariants \nAbstract: Although the detection of invariant structure in a given set of input patterns is vital to many recognition tasks, connectionist learning rules tend to focus on directions of high variance (principal components). The prediction paradigm is often used to reconcile this dichotomy; here we suggest a more direct approach to invariant learning based on an anti-Hebbian learning rule. An unsupervised two-layer network implementing this method in a competitive setting learns to extract coherent depth information from random-dot stereograms.",
523 "neighbors": [
524 576,
525 747
526 ],
527 "mask": "Train"
528 },
529 {
530 "node_id": 44,
531 "label": 0,
532 "text": "Title: Competitive Anti-Hebbian Learning of Invariants \nAbstract: Instance-based learning methods explicitly remember all the data that they receive. They usually have no training phase, and only at prediction time do they perform computation. Then, they take a query, search the database for similar datapoints and build an on-line local model (such as a local average or local regression) with which to predict an output value. In this paper we review the advantages of instance based methods for autonomous systems, but we also note the ensuing cost: hopelessly slow computation as the database grows large. We present and evaluate a new way of structuring a database and a new algorithm for accessing it that maintains the advantages of instance-based learning. Earlier attempts to combat the cost of instance-based learning have sacrificed the explicit retention of all data, or been applicable only to instance-based predictions based on a small number of near neighbors or have had to re-introduce an explicit training phase in the form of an interpolative data structure. Our approach builds a multiresolution data structure to summarize the database of experiences at all resolutions of interest simultaneously. This permits us to query the database with the same exibility as a conventional linear search, but at greatly reduced computational cost.",
533 "neighbors": [
534 88,
535 686,
536 2428
537 ],
538 "mask": "Validation"
539 },
540 {
541 "node_id": 45,
542 "label": 4,
543 "text": "Title: Acting under Uncertainty: Discrete Bayesian Models for Mobile-Robot Navigation \nAbstract: Discrete Bayesian models have been used to model uncertainty for mobile-robot navigation, but the question of how actions should be chosen remains largely unexplored. This paper presents the optimal solution to the problem, formulated as a partially observable Markov decision process. Since solving for the optimal control policy is intractable, in general, it goes on to explore a variety of heuristic control strategies. The control strategies are compared experimentally, both in simulation and in runs on a robot. ",
544 "neighbors": [
545 220,
546 490,
547 492,
548 1459
549 ],
550 "mask": "Train"
551 },
552 {
553 "node_id": 46,
554 "label": 2,
555 "text": "Title: The Pandemonium System of Reflective Agents \nAbstract: In IEEE Transactions on Neural Networks, 7(1):97-106, 1996 Also available as GMD report #794 ",
556 "neighbors": [
557 238,
558 301,
559 489,
560 867
561 ],
562 "mask": "Validation"
563 },
564 {
565 "node_id": 47,
566 "label": 0,
567 "text": "Title: An Implementation and Experiment with the Nested Generalized Exemplars Algorithm \nAbstract: This NRL NCARAI technical note (AIC-95-003) describes work with Salzberg's (1991) NGE. I recently implemented this algorithm and have run a few case studies. The purpose of this note is to publicize this implementation and note a curious result while using it. This implementation of NGE is available at under my WWW address ",
568 "neighbors": [
569 87
570 ],
571 "mask": "Train"
572 },
573 {
574 "node_id": 48,
575 "label": 3,
576 "text": "Title: Sampling from Multimodal Distributions Using Tempered Transitions \nAbstract: Technical Report No. 9421, Department of Statistics, University of Toronto Abstract. I present a new Markov chain sampling method appropriate for distributions with isolated modes. Like the recently-developed method of \"simulated tempering\", the \"tempered transition\" method uses a series of distributions that interpolate between the distribution of interest and a distribution for which sampling is easier. The new method has the advantage that it does not require approximate values for the normalizing constants of these distributions, which are needed for simulated tempering, and can be tedious to estimate. Simulated tempering performs a random walk along the series of distributions used. In contrast, the tempered transitions of the new method move systematically from the desired distribution, to the easily-sampled distribution, and back to the desired distribution. This systematic movement avoids the inefficiency of a random walk, an advantage that unfortunately is cancelled by an increase in the number of interpolating distributions required. Because of this, the sampling efficiency of the tempered transition method in simple problems is similar to that of simulated tempering. On more complex distributions, however, simulated tempering and tempered transitions may perform differently. Which is better depends on the ways in which the interpolating distributions are \"deceptive\". ",
577 "neighbors": [
578 725,
579 1783,
580 2456,
581 2682
582 ],
583 "mask": "Train"
584 },
585 {
586 "node_id": 49,
587 "label": 0,
588 "text": "Title: Abstract \nAbstract: We describe an ongoing project to develop an adaptive training system (ATS) that dynamically models a students learning processes and can provide specialized tutoring adapted to a students knowledge state and learning style. The student modeling component of the ATS, ML-Modeler, uses machine learning (ML) techniques to emulate the students novice-to-expert transition. ML-Modeler infers which learning methods the student has used to reach the current knowledge state by comparing the students solution trace to an expert solution and generating plausible hypotheses about what misconceptions and errors the student has made. A case-based approach is used to generate hypotheses through incorrectly applying analogy, overgeneralization, and overspecialization. The student and expert models use a network-based representation that includes abstract concepts and relationships as well as strategies for problem solving. Fuzzy methods are used to represent the uncertainty in the student model. This paper describes the design of the ATS and ML-Modeler, and gives a detailed example of how the system would model and tutor the student in a typical session. The domain we use for this example is high-school level chemistry. ",
589 "neighbors": [
590 581,
591 643
592 ],
593 "mask": "Validation"
594 },
595 {
596 "node_id": 50,
597 "label": 0,
598 "text": "Title: Abstract \nAbstract: Metacognition addresses the issues of knowledge about cognition and regulating cognition. We argue that the regulation process should be improved with growing experience. Therefore mental models are needed which facilitate the re-use of previous regulation processes. We will satisfy this requirement by describing a case-based approach to Introspection Planning which utilises previous experience obtained during reasoning at the meta-level and at the object level. The introspection plans used in this approach support various metacognitive tasks which are identified by the generation of self-questions. As an example of introspection planning, the metacognitive behaviour of our system, IULIAN, is described. ",
599 "neighbors": [
600 150,
601 581,
602 583,
603 643
604 ],
605 "mask": "Train"
606 },
607 {
608 "node_id": 51,
609 "label": 3,
610 "text": "Title: An Alternative Markov Property for Chain Graphs \nAbstract: Graphical Markov models use graphs, either undirected, directed, or mixed, to represent possible dependences among statistical variables. Applications of undirected graphs (UDGs) include models for spatial dependence and image analysis, while acyclic directed graphs (ADGs), which are especially convenient for statistical analysis, arise in such fields as genetics and psychometrics and as models for expert systems and Bayesian belief networks. Lauritzen, Wer-muth, and Frydenberg (LWF) introduced a Markov property for chain graphs, which are mixed graphs that can be used to represent simultaneously both causal and associative dependencies and which include both UDGs and ADGs as special cases. In this paper an alternative Markov property (AMP) for chain graphs is introduced, which in some ways is a more direct extension of the ADG Markov property than is the LWF property for chain graph.",
611 "neighbors": [
612 645,
613 772,
614 2559
615 ],
616 "mask": "Validation"
617 },
618 {
619 "node_id": 52,
620 "label": 6,
621 "text": "Title: Theory Revision in Fault Hierarchies \nAbstract: The fault hierarchy representation is widely used in expert systems for the diagnosis of complex mechanical devices. On the assumption that an appropriate bias for a knowledge representation language is also an appropriate bias for learning in this domain, we have developed a theory revision method that operates directly on a fault hierarchy. This task presents several challenges: A typical training instance is missing most feature values, and the pattern of missing features is significant, rather than merely an effect of noise. Moreover, the accuracy of a candidate theory is measured by considering both the sequence of tests required to arrive at a diagnosis and its agreement with the diagnostic endpoints provided by an expert. This paper first describes the algorithm for theory revision of fault hierarchies that was designed to address these challenges, then discusses its application in knowledge base maintenance and reports on experiments that use to revise a fielded diagnostic system. ",
622 "neighbors": [
623 228,
624 430,
625 478,
626 2487,
627 2580
628 ],
629 "mask": "Train"
630 },
631 {
632 "node_id": 53,
633 "label": 1,
634 "text": "Title: DISTRIBUTED GENETIC ALGORITHMS FOR PARTITIONING UNIFORM GRIDS \nAbstract: The fault hierarchy representation is widely used in expert systems for the diagnosis of complex mechanical devices. On the assumption that an appropriate bias for a knowledge representation language is also an appropriate bias for learning in this domain, we have developed a theory revision method that operates directly on a fault hierarchy. This task presents several challenges: A typical training instance is missing most feature values, and the pattern of missing features is significant, rather than merely an effect of noise. Moreover, the accuracy of a candidate theory is measured by considering both the sequence of tests required to arrive at a diagnosis and its agreement with the diagnostic endpoints provided by an expert. This paper first describes the algorithm for theory revision of fault hierarchies that was designed to address these challenges, then discusses its application in knowledge base maintenance and reports on experiments that use to revise a fielded diagnostic system. ",
635 "neighbors": [
636 243,
637 803,
638 1439,
639 1563
640 ],
641 "mask": "Train"
642 },
643 {
644 "node_id": 54,
645 "label": 6,
646 "text": "Title: A Competitive Approach to Game Learning \nAbstract: Machine learning of game strategies has often depended on competitive methods that continually develop new strategies capable of defeating previous ones. We use a very inclusive definition of game and consider a framework within which a competitive algorithm makes repeated use of a strategy learning component that can learn strategies which defeat a given set of opponents. We describe game learning in terms of sets H and X of first and second player strategies, and connect the model with more familiar models of concept learning. We show the importance of the ideas of teaching set [20] and specification number [19] k in this new context. The performance of several competitive algorithms is investigated, using both worst-case and randomized strategy learning algorithms. Our central result (Theorem 4) is a competitive algorithm that solves games in a total number of strategies polynomial in lg(jHj), lg(jX j), and k. Its use is demonstrated, including an application in concept learning with a new kind of counterexample oracle. We conclude with a complexity analysis of game learning, and list a number of new questions arising from this work. ",
647 "neighbors": [
648 523,
649 615,
650 712,
651 1687,
652 2334
653 ],
654 "mask": "Train"
655 },
656 {
657 "node_id": 55,
658 "label": 1,
659 "text": "Title: A Comparison of Selection Schemes used in Genetic Algorithms \nAbstract: TIK-Report Nr. 11, December 1995 Version 2 (2. Edition) ",
660 "neighbors": [
661 163,
662 361,
663 844,
664 1784,
665 1832,
666 1905
667 ],
668 "mask": "Train"
669 },
670 {
671 "node_id": 56,
672 "label": 6,
673 "text": "Title: Self bounding learning algorithms \nAbstract: Most of the work which attempts to give bounds on the generalization error of the hypothesis generated by a learning algorithm is based on methods from the theory of uniform convergence. These bounds are a-priori bounds that hold for any distribution of examples and are calculated before any data is observed. In this paper we propose a different approach for bounding the generalization error after the data has been observed. A self-bounding learning algorithm is an algorithm which, in addition to the hypothesis that it outputs, outputs a reliable upper bound on the generalization error of this hypothesis. We first explore the idea in the statistical query learning framework of Kearns [10]. After that we give an explicit self bounding algorithm for learning algorithms that are based on local search.",
674 "neighbors": [
675 778,
676 967,
677 1027
678 ],
679 "mask": "Train"
680 },
681 {
682 "node_id": 57,
683 "label": 4,
684 "text": "Title: Markov Decision Processes in Large State Spaces \nAbstract: In this paper we propose a new framework for studying Markov decision processes (MDPs), based on ideas from statistical mechanics. The goal of learning in MDPs is to find a policy that yields the maximum expected return over time. In choosing policies, agents must therefore weigh the prospects of short-term versus long-term gains. We study a simple MDP in which the agent must constantly decide between exploratory jumps and local reward mining in state space. The number of policies to choose from grows exponentially with the size of the state space, N . We view the expected returns as defining an energy landscape over policy space. Methods from statistical mechanics are used to analyze this landscape in the thermodynamic limit N ! 1. We calculate the overall distribution of expected returns, as well as the distribution of returns for policies at a fixed Hamming distance from the optimal one. We briefly discuss the problem of learning optimal policies from empirical estimates of the expected return. As a first step, we relate our findings for the entropy to the limit of high-temperature learning. Numerical simulations support the theoretical results. ",
685 "neighbors": [
686 306,
687 552,
688 565,
689 967,
690 1459
691 ],
692 "mask": "Train"
693 },
694 {
695 "node_id": 58,
696 "label": 2,
697 "text": "Title: Neural Networks with Quadratic VC Dimension \nAbstract: This paper shows that neural networks which use continuous activation functions have VC dimension at least as large as the square of the number of weights w. This result settles a long-standing open question, namely whether the well-known O(w log w) bound, known for hard-threshold nets, also held for more general sigmoidal nets. Implications for the number of samples needed for valid generalization are discussed. ",
698 "neighbors": [
699 536,
700 990,
701 1891,
702 2495
703 ],
704 "mask": "Train"
705 },
706 {
707 "node_id": 59,
708 "label": 2,
709 "text": "Title: SELF-ADAPTIVE NEURAL NETWORKS FOR BLIND SEPARATION OF SOURCES \nAbstract: Novel on-line learning algorithms with self adaptive learning rates (parameters) for blind separation of signals are proposed. The main motivation for development of new learning rules is to improve convergence speed and to reduce cross-talking, especially for non-stationary signals. Furthermore, we have discovered that under some conditions the proposed neural network models with associated learning algorithms exhibit a random switch of attention, i.e. they have ability of chaotic or random switching or cross-over of output signals in such way that a specified separated signal may appear at various outputs at different time windows. Validity, performance and dynamic properties of the proposed learning algorithms are investigated by computer simulation experiments. ",
710 "neighbors": [
711 570,
712 576,
713 839,
714 872,
715 874,
716 1520,
717 1709
718 ],
719 "mask": "Validation"
720 },
721 {
722 "node_id": 60,
723 "label": 4,
724 "text": "Title: The Efficient Learning of Multiple Task Sequences \nAbstract: I present a modular network architecture and a learning algorithm based on incremental dynamic programming that allows a single learning agent to learn to solve multiple Markovian decision tasks (MDTs) with significant transfer of learning across the tasks. I consider a class of MDTs, called composite tasks, formed by temporally concatenating a number of simpler, elemental MDTs. The architecture is trained on a set of composite and elemental MDTs. The temporal structure of a composite task is assumed to be unknown and the architecture learns to produce a temporal decomposition. It is shown that under certain conditions the solution of a composite MDT can be constructed by computationally inexpensive modifications of the solutions of its constituent elemental MDTs.",
725 "neighbors": [
726 252,
727 552,
728 562,
729 565,
730 688
731 ],
732 "mask": "Train"
733 },
734 {
735 "node_id": 61,
736 "label": 0,
737 "text": "Title: Program Synthesis and Transformation Techniques for Simpuation, Optimization and Constraint Satisfaction Deductive Synthesis of Numerical\nAbstract: Scientists and engineers face recurring problems of constructing, testing and modifying numerical simulation programs. The process of coding and revising such simulators is extremely time-consuming, because they are almost always written in conventional programming languages. Scientists and engineers can therefore benefit from software that facilitates construction of programs for simulating physical systems. Our research adapts the methodology of deductive program synthesis to the problem of constructing numerical simulation codes. We have focused on simulators that can be represented as second order functional programs composed of numerical integration and root extraction routines. We have developed a system that uses first order Horn logic to synthesize numerical simulators built from these components. Our approach is based on two ideas: First, we axiomatize only the relationship between integration and differentiation. We neither attempt nor require a complete axiomatization of mathematical analysis. Second, our system uses a representation in which functions are reified as objects. Function objects are encoded as lambda expressions. Our knowledge base includes an axiomatization of term equality in the lambda calculus. It also includes axioms defining the semantics of numerical integration and root extraction routines. We use depth bounded SLD resolution to construct proofs and synthesize programs. Our system has successfully constructed numerical simulators for computational design of jet engine nozzles and sailing yachts, among others. Our results demonstrate that deductive synthesis techniques can be used to construct numerical simulation programs for realistic applications (Ellman and Murata 1998). Automatic design optimization is highly sensitive to problem formulation. The choice of objective function, constraints and design parameters can dramatically impact the computational cost of optimization and the quality of the resulting design. The best formulation varies from one application to another. A design engineer will usually not know the best formulation in advance. In order to address this problem, we have developed a system that supports interactive formulation, testing and reformulation of design optimization strategies. Our system includes an executable, data-flow language for representing optimization strategies. The language allows an engineer to define multiple stages of optimization, each using different approximations of the objective and constraints or different abstractions of the design space. We have also developed a set of transformations that reformulate strategies represented in our language. The transformations can approximate objective and constraint functions, abstract or reparameterize search spaces, or divide an optimization process into multiple stages. The system is applicable in principle to any design problem that can be expressed in terms of constrained op ",
738 "neighbors": [
739 240
740 ],
741 "mask": "Train"
742 },
743 {
744 "node_id": 62,
745 "label": 3,
746 "text": "Title: Context-Specific Independence in Bayesian Networks \nAbstract: Bayesiannetworks provide a languagefor qualitatively representing the conditional independence properties of a distribution. This allows a natural and compact representation of the distribution, eases knowledge acquisition, and supports effective inference algorithms. It is well-known, however, that there are certain independencies that we cannot capture qualitatively within the Bayesian network structure: independencies that hold only in certain contexts, i.e., given a specific assignment of values to certain variables. In this paper, we propose a formal notion of context-specific independence (CSI), based on regularities in the conditional probability tables (CPTs) at a node. We present a technique, analogous to (and based on) d-separation, for determining when such independence holds in a given network. We then focus on a particular qualitative representation schemetree-structured CPTs for capturing CSI. We suggest ways in which this representation can be used to support effective inference algorithms. In particular, we present a structural decomposition of the resulting network which can improve the performance of clustering algorithms, and an alternative algorithm based on cutset conditioning.",
747 "neighbors": [
748 324,
749 332,
750 423,
751 945,
752 2425,
753 2474,
754 2566
755 ],
756 "mask": "Validation"
757 },
758 {
759 "node_id": 63,
760 "label": 4,
761 "text": "Title: Machine Learning, Reinforcement Learning with Replacing Eligibility Traces \nAbstract: The eligibility trace is one of the basic mechanisms used in reinforcement learning to handle delayed reward. In this paper we introduce a new kind of eligibility trace, the replacing trace, analyze it theoretically, and show that it results in faster, more reliable learning than the conventional trace. Both kinds of trace assign credit to prior events according to how recently they occurred, but only the conventional trace gives greater credit to repeated events. Our analysis is for conventional and replace-trace versions of the o*ine TD(1) algorithm applied to undiscounted absorbing Markov chains. First, we show that these methods converge under repeated presentations of the training set to the same predictions as two well known Monte Carlo methods. We then analyze the relative efficiency of the two Monte Carlo methods. We show that the method corresponding to conventional TD is biased, whereas the method corresponding to replace-trace TD is unbiased. In addition, we show that the method corresponding to replacing traces is closely related to the maximum likelihood solution for these tasks, and that its mean squared error is always lower in the long run. Computational results confirm these analyses and show that they are applicable more generally. In particular, we show that replacing traces significantly improve performance and reduce parameter sensitivity on the \"Mountain-Car\" task, a full reinforcement-learning problem with a continuous state space, when using a feature-based function approximator. ",
762 "neighbors": [
763 153,
764 210,
765 739,
766 1546
767 ],
768 "mask": "Test"
769 },
770 {
771 "node_id": 64,
772 "label": 0,
773 "text": "Title: Integrating Creativity and Reading: A Functional Approach \nAbstract: Reading has been studied for decades by a variety of cognitive disciplines, yet no theories exist which sufficiently describe and explain how people accomplish the complete task of reading real-world texts. In particular, a type of knowledge intensive reading known as creative reading has been largely ignored by the past research. We argue that creative reading is an aspect of practically all reading experiences; as a result, any theory which overlooks this will be insufficient. We have built on results from psychology, artificial intelligence, and education in order to produce a functional theory of the complete reading process. The overall framework describes the set of tasks necessary for reading to be performed. Within this framework, we have developed a theory of creative reading. The theory is implemented in the ISAAC (Integrated Story Analysis And Creativity) system, a reading system which reads science fiction stories. ",
774 "neighbors": [
775 284,
776 289,
777 486,
778 583
779 ],
780 "mask": "Train"
781 },
782 {
783 "node_id": 65,
784 "label": 1,
785 "text": "Title: Integrating Creativity and Reading: A Functional Approach \nAbstract: dvitps ERROR: reno98b.dvi @ puccini.rutgers.edu Certain fonts that you requested in your dvi file could not be found on the system. In order to print your document, other fonts that are installed were substituted for these missing fonts. Below is a list of the substitutions that were made. /usr/local/lib/fonts/gf/cmbx12.518pk substituted for cmbx12.519pk ",
786 "neighbors": [
787 743,
788 744
789 ],
790 "mask": "Train"
791 },
792 {
793 "node_id": 66,
794 "label": 0,
795 "text": "Title: (1994); Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. Case-Based Reasoning: Foundational Issues, Methodological\nAbstract: Case-based reasoning is a recent approach to problem solving and learning that has got a lot of attention over the last few years. Originating in the US, the basic idea and underlying theories have spread to other continents, and we are now within a period of highly active research in case-based reasoning in Europe, as well. This paper gives an overview of the foundational issues related to case-based reasoning, describes some of the leading methodological approaches within the field, and exemplifies the current state through pointers to some systems. Initially, a general framework is defined, to which the subsequent descriptions and discussions will refer. The framework is influenced by recent methodologies for knowledge level descriptions of intelligent systems. The methods for case retrieval, reuse, solution testing, and learning are summarized, and their actual realization is discussed in the light of a few example systems that represent different CBR approaches. We also discuss the role of case-based methods as one type of reasoning and learning method within an integrated system architecture. ",
796 "neighbors": [
797 34,
798 69,
799 149,
800 183,
801 215,
802 288,
803 1248,
804 1531,
805 1698,
806 2122,
807 2157,
808 2294,
809 2310,
810 2359,
811 2441,
812 2520
813 ],
814 "mask": "Train"
815 },
816 {
817 "node_id": 67,
818 "label": 3,
819 "text": "Title: Updates and Counterfactuals \nAbstract: We study the problem of combining updates |a special instance of theory change| and counterfactual conditionals in propositional knowledgebases. Intuitively, an update means that the world described by the knowledgebase has changed. This is opposed to revisions |another instance of theory change| where our knowledge about a static world changes. A counterfactual implication is a statement of the form `If A were the case, then B would also be the case', where the negation of A may be derivable from our current knowledge. We present a decidable logic, called VCU 2 , that has both update and counterfactual implication as connectives in the object language. Our update operator is a generalization of operators previously proposed and studied in the literature. We show that our operator satisfies certain postulates set forth for any reasonable update. The logic VCU 2 is an extension of D. K. Lewis' logic VCU for counterfactual conditionals. The semantics of VCU 2 is that of a multimodal propositional calculus, and is based on possible worlds. The infamous Ramsey Rule becomes a derivation rule in our sound and complete axiomatization. We then show that Gardenfors' Triviality Theorem, about the impossibility to combine theory change and counterfactual conditionals via the Ramsey Rule, does not hold in our logic. It is thus seen that the Triviality Theorem applies only to revision operators, not to updates. fl A preliminary version of this paper was presented at the Second International Conference on Principles of Knowledge Representation and Reasoning, Cambridge, Massachusetts, April 22-25, 1991. The work was partially performed while the author was visiting the Department of Computer Science at the University of Toronto. ",
820 "neighbors": [
821 729
822 ],
823 "mask": "Test"
824 },
825 {
826 "node_id": 68,
827 "label": 2,
828 "text": "Title: DISCOVERING NEURAL NETS WITH LOW KOLMOGOROV COMPLEXITY AND HIGH GENERALIZATION CAPABILITY Neural Networks 10(5):857-873, 1997 \nAbstract: Many neural net learning algorithms aim at finding \"simple\" nets to explain training data. The expectation is: the \"simpler\" the networks, the better the generalization on test data (! Occam's razor). Previous implementations, however, use measures for \"simplicity\" that lack the power, universality and elegance of those based on Kolmogorov complexity and Solomonoff's algorithmic probability. Likewise, most previous approaches (especially those of the \"Bayesian\" kind) suffer from the problem of choosing appropriate priors. This paper addresses both issues. It first reviews some basic concepts of algorithmic complexity theory relevant to machine learning, and how the Solomonoff-Levin distribution (or universal prior) deals with the prior problem. The universal prior leads to a probabilistic method for finding \"algorithmically simple\" problem solutions with high generalization capability. The method is based on Levin complexity (a time-bounded generalization of Kolmogorov complexity) and inspired by Levin's optimal universal search algorithm. For a given problem, solution candidates are computed by efficient \"self-sizing\" programs that influence their own runtime and storage size. The probabilistic search algorithm finds the \"good\" programs (the ones quickly computing algorithmically probable solutions fitting the training data). Simulations focus on the task of discovering \"algorithmically simple\" neural networks with low Kolmogorov complexity and high generalization capability. It is demonstrated that the method, at least with certain toy problems where it is computationally feasible, can lead to generalization results unmatchable by previous neural net algorithms. Much remains do be done, however, to make large scale applications and \"incremental learning\" feasible.",
829 "neighbors": [
830 522,
831 979,
832 1632,
833 1825,
834 1845,
835 1979,
836 2007
837 ],
838 "mask": "Train"
839 },
840 {
841 "node_id": 69,
842 "label": 0,
843 "text": "Title: SaxEx a case-based reasoning system for generating expressive musical performances \nAbstract: We have studied the problem of generating expressive musical performances in the context of tenor saxophone interpretations. We have done several recordings of a tenor sax playing different Jazz ballads with different degrees of expressiveness including an inexpressive interpretation of each ballad. These recordings are analyzed, using SMS spectral modeling techniques, to extract information related to several expressive parameters. This set of parameters and the scores constitute the set of cases (examples) of a case-based system. From this set of cases, the system infers a set of possible expressive transformations for a given new phrase applying similarity criteria, based on background musical knowledge, between this new phrase and the set of cases. Finally, SaxEx applies the inferred expressive transformations to the new phrase using the synthesis capabilities of SMS.",
844 "neighbors": [
845 66
846 ],
847 "mask": "Train"
848 },
849 {
850 "node_id": 70,
851 "label": 6,
852 "text": "Title: Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods \nAbstract: One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance decomposition. ",
853 "neighbors": [
854 255,
855 931,
856 999,
857 1521,
858 1692,
859 1986
860 ],
861 "mask": "Train"
862 },
863 {
864 "node_id": 71,
865 "label": 3,
866 "text": "Title: Supervised learning from incomplete data via an EM approach \nAbstract: Real-world learning tasks may involve high-dimensional data sets with arbitrary patterns of missing data. In this paper we present a framework based on maximum likelihood density estimation for learning from such data sets. We use mixture models for the density estimates and make two distinct appeals to the Expectation-Maximization (EM) principle (Dempster et al., 1977) in deriving a learning algorithm|EM is used both for the estimation of mixture components and for coping with missing data. The resulting algorithm is applicable to a wide range of supervised as well as unsupervised learning problems. Results from a classification benchmark|the iris data set|are presented.",
867 "neighbors": [
868 74,
869 661,
870 677,
871 929,
872 1559,
873 1641,
874 1923,
875 2442
876 ],
877 "mask": "Train"
878 },
879 {
880 "node_id": 72,
881 "label": 2,
882 "text": "Title: SCRIPT RECOGNITION WITH HIERARCHICAL FEATURE MAPS \nAbstract: The hierarchical feature map system recognizes an input story as an instance of a particular script by classifying it at three levels: scripts, tracks and role bindings. The recognition taxonomy, i.e. the breakdown of each script into the tracks and roles, is extracted automatically and independently for each script from examples of script instantiations in an unsupervised self-organizing process. The process resembles human learning in that the differentiation of the most frequently encountered scripts become gradually the most detailed. The resulting structure is a hierachical pyramid of feature maps. The hierarchy visualizes the taxonomy and the maps lay out the topology of each level. The number of input lines and the self-organization time are considerably reduced compared to the ordinary single-level feature mapping. The system can recognize incomplete stories and recover the missing events. The taxonomy also serves as memory organization for script-based episodic memory. The maps assign a unique memory location for each script instantiation. The most salient parts of the input data are separated and most resources are concentrated on representing them accurately. ",
883 "neighbors": [
884 202,
885 745,
886 747,
887 771
888 ],
889 "mask": "Train"
890 },
891 {
892 "node_id": 73,
893 "label": 4,
894 "text": "Title: LEARNING TO GENERATE ARTIFICIAL FOVEA TRAJECTORIES FOR TARGET DETECTION \nAbstract: It is shown how `static' neural approaches to adaptive target detection can be replaced by a more efficient and more sequential alternative. The latter is inspired by the observation that biological systems employ sequential eye-movements for pattern recognition. A system is described which builds an adaptive model of the time-varying inputs of an artificial fovea controlled by an adaptive neural controller. The controller uses the adaptive model for learning the sequential generation of fovea trajectories causing the fovea to move to a target in a visual scene. The system also learns to track moving targets. No teacher provides the desired activations of `eye-muscles' at various times. The only goal information is the shape of the target. Since the task is a `reward-only-at-goal' task , it involves a complex temporal credit assignment problem. Some implications for adaptive attentive systems in general are discussed. ",
895 "neighbors": [
896 747
897 ],
898 "mask": "Train"
899 },
900 {
901 "node_id": 74,
902 "label": 3,
903 "text": "Title: Hierarchical Mixtures of Experts and the EM Algorithm \nAbstract: We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. ",
904 "neighbors": [
905 19,
906 71,
907 154,
908 193,
909 252,
910 263,
911 310,
912 345,
913 377,
914 505,
915 511,
916 547,
917 604,
918 622,
919 661,
920 680,
921 787,
922 867,
923 871,
924 881,
925 906,
926 949,
927 975,
928 987,
929 1017,
930 1024,
931 1103,
932 1220,
933 1634,
934 1676,
935 1928,
936 2013,
937 2124,
938 2266,
939 2284,
940 2335,
941 2390,
942 2421,
943 2513,
944 2654
945 ],
946 "mask": "Validation"
947 },
948 {
949 "node_id": 75,
950 "label": 0,
951 "text": "Title: A Memory Model for Case Retrieval by Activation Passing \nAbstract: We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain. This report describes research done at the Dept. of Brain and Cognitive Sciences, the Center for Biological and Computational Learning, and the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for CBCL is provided in part by a grant from the NSF (ASC-9217041). Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Dept. of Defense. The authors were supported by a grant from the McDonnell-Pew Foundation, by a grant from ATR Human Information Processing Research Laboratories, by a grant from Siemens Corporation, by by grant IRI-9013991 from the National Science Foundation, by grant N00014-90-J-1942 from the Office of Naval Research, and by NSF grant ECS-9216531 to support an Initiative in Intelligent Control at MIT. Michael I. Jordan is a NSF Presidential Young Investigator. ",
952 "neighbors": [
953 288,
954 1123,
955 1354,
956 1854,
957 1855,
958 2122,
959 2299
960 ],
961 "mask": "Train"
962 },
963 {
964 "node_id": 76,
965 "label": 3,
966 "text": "Title: A VIEW OF THE EM ALGORITHM THAT JUSTIFIES INCREMENTAL, SPARSE, AND OTHER VARIANTS \nAbstract: The EM algorithm performs maximum likelihood estimation for data in which some variables are unobserved. We present a function that resembles negative free energy and show that the M step maximizes this function with respect to the model parameters and the E step maximizes it with respect to the distribution over the unobserved variables. From this perspective, it is easy to justify an incremental variant of the EM algorithm in which the distribution for only one of the unobserved variables is recalculated in each E step. This variant is shown empirically to give faster convergence in a mixture estimation problem. A variant of the algorithm that exploits sparse conditional distributions is also described, and a wide range of other variant algorithms are also seen to be possible. ",
967 "neighbors": [
968 131,
969 181,
970 250,
971 345,
972 392,
973 518,
974 661,
975 694,
976 869,
977 975,
978 1128,
979 1548,
980 1934,
981 2327,
982 2390,
983 2532
984 ],
985 "mask": "Train"
986 },
987 {
988 "node_id": 77,
989 "label": 2,
990 "text": "Title: Synchronization and Desynchronization in a Network of Locally Coupled Wilson-Cowan Oscillators \nAbstract: A network of Wilson-Cowan oscillators is constructed, and its emergent properties of synchronization and desynchronization are investigated by both computer simulation and formal analysis. The network is a two-dimensional matrix, where each oscillator is coupled only to its neighbors. We show analytically that a chain of locally coupled oscillators (the piece-wise linear approximation to the Wilson-Cowan oscillator) synchronizes, and present a technique to rapidly entrain finite numbers of oscillators. The coupling strengths change on a fast time scale based on a Hebbian rule. A global separator is introduced which receives input from and sends feedback to each oscillator in the matrix. The global separator is used to desynchronize different oscillator groups. Unlike many other models, the properties of this network emerge from local connections, that preserve spatial relationships among components, and are critical for encoding Gestalt principles of feature grouping. The ability to synchronize and desynchronize oscillator groups within this network offers a promising approach for pattern segmentation and figure/ground segregation based on oscillatory correlation. ",
991 "neighbors": [
992 337
993 ],
994 "mask": "Train"
995 },
996 {
997 "node_id": 78,
998 "label": 6,
999 "text": "Title: Probabilistic Networks: New Models and New Methods \nAbstract: In this paper I describe the implementation of a probabilistic regression model in BUGS. BUGS is a program that carries out Bayesian inference on statistical problems using a simulation technique known as Gibbs sampling. It is possible to implement surprisingly complex regression models in this environment. I demonstrate the simultaneous inference of an interpolant and an input-dependent noise level. ",
1000 "neighbors": [
1001 157,
1002 214,
1003 469,
1004 560,
1005 766,
1006 2681
1007 ],
1008 "mask": "Validation"
1009 },
1010 {
1011 "node_id": 79,
1012 "label": 6,
1013 "text": "Title: A hierarchical ensemble of decision trees applied to classifying data from a psychological experiment \nAbstract: Classifying by hand complex data coming from psychology experiments can be a long and difficult task, because of the quantity of data to classify and the amount of training it may require. One way to alleviate this problem is to use machine learning techniques. We built a classifier based on decision trees that reproduces the classifying process used by two humans on a sample of data and that learns how to classify unseen data. The automatic classifier proved to be more accurate, more constant and much faster than classification by hand. ",
1014 "neighbors": [
1015 438,
1016 2207
1017 ],
1018 "mask": "Validation"
1019 },
1020 {
1021 "node_id": 80,
1022 "label": 2,
1023 "text": "Title: Neural Network Implementation in SAS R Software \nAbstract: The estimation or training methods in the neural network literature are usually some simple form of gradient descent algorithm suitable for implementation in hardware using massively parallel computations. For ordinary computers that are not massively parallel, optimization algorithms such as those in several SAS procedures are usually far more efficient. This talk shows how to fit neural networks using SAS/OR R fl , SAS/ETS R fl , and SAS/STAT R fl software. ",
1024 "neighbors": [
1025 15,
1026 2044
1027 ],
1028 "mask": "Train"
1029 },
1030 {
1031 "node_id": 81,
1032 "label": 3,
1033 "text": "Title: A Modification to Evidential Probability \nAbstract: Selecting the right reference class and the right interval when faced with conflicting candidates and no possibility of establishing subset style dominance has been a problem for Kyburg's Evidential Probability system. Various methods have been proposed by Loui and Kyburg to solve this problem in a way that is both intuitively appealing and justifiable within Kyburg's framework. The scheme proposed in this paper leads to stronger statistical assertions without sacrificing too much of the intuitive appeal of Kyburg's latest proposal. ",
1034 "neighbors": [
1035 647
1036 ],
1037 "mask": "Validation"
1038 },
1039 {
1040 "node_id": 82,
1041 "label": 4,
1042 "text": "Title: A Reinforcement Learning Approach to Job-shop Scheduling \nAbstract: We apply reinforcement learning methods to learn domain-specific heuristics for job shop scheduling. A repair-based scheduler starts with a critical-path schedule and incrementally repairs constraint violations with the goal of finding a short conflict-free schedule. The temporal difference algorithm T D() is applied to train a neural network to learn a heuristic evaluation function over states. This evaluation function is used by a one-step looka-head search procedure to find good solutions to new scheduling problems. We evaluate this approach on synthetic problems and on problems from a NASA space shuttle payload processing task. The evaluation function is trained on problems involving a small number of jobs and then tested on larger problems. The TD sched-uler performs better than the best known existing algorithm for this task|Zweben's iterative repair method based on simulated annealing. The results suggest that reinforcement learning can provide a new method for constructing high-performance scheduling systems.",
1043 "neighbors": [
1044 39,
1045 239,
1046 295,
1047 305,
1048 410,
1049 548,
1050 565,
1051 1378,
1052 1440,
1053 1553
1054 ],
1055 "mask": "Train"
1056 },
1057 {
1058 "node_id": 83,
1059 "label": 2,
1060 "text": "Title: A Neural Network Pole Balancer that Learns and Operates on a Real Robot in Real Time \nAbstract: A neural network approach to the classic inverted pendulum task is presented. This task is the task of keeping a rigid pole, hinged to a cart and free to fall in a plane, in a roughly vertical orientation by moving the cart horizontally in the plane while keeping the cart within some maximum distance of its starting position. This task constitutes a difficult control problem if the parameters of the cart-pole system are not known precisely or are variable. It also forms the basis of an even more complex control-learning problem if the controller must learn the proper actions for successfully balancing the pole given only the current state of the system and a failure signal when the pole angle from the vertical becomes too great or the cart exceeds one of the boundaries placed on its position. The approach presented is demonstrated to be effective for the real-time control of a small, self-contained mini-robot, specially outfitted for the task. Origins and details of the learning scheme, specifics of the mini-robot hardware, and results of actual learning trials are presented. ",
1061 "neighbors": [
1062 294,
1063 747
1064 ],
1065 "mask": "Validation"
1066 },
1067 {
1068 "node_id": 84,
1069 "label": 3,
1070 "text": "Title: Approximate Bayes Factors and Accounting for Model Uncertainty in Generalized Linear Models \nAbstract: Technical Report no. 255 Department of Statistics, University of Washington August 1993; Revised March 1994 ",
1071 "neighbors": [
1072 12,
1073 27,
1074 155,
1075 347,
1076 715,
1077 950,
1078 998,
1079 1240,
1080 1241,
1081 1347,
1082 1550,
1083 1803
1084 ],
1085 "mask": "Test"
1086 },
1087 {
1088 "node_id": 85,
1089 "label": 4,
1090 "text": "Title: Q-Learning with Hidden-Unit Restarting \nAbstract: Platt's resource-allocation network (RAN) (Platt, 1991a, 1991b) is modified for a reinforcement-learning paradigm and to \"restart\" existing hidden units rather than adding new units. After restarting, units continue to learn via back-propagation. The resulting restart algorithm is tested in a Q-learning network that learns to solve an inverted pendulum problem. Solutions are found faster on average with the restart algorithm than without it.",
1091 "neighbors": [
1092 294,
1093 425,
1094 465,
1095 552,
1096 565,
1097 2368
1098 ],
1099 "mask": "Train"
1100 },
1101 {
1102 "node_id": 86,
1103 "label": 5,
1104 "text": "Title: THE EXPANDABLE SPLIT WINDOW PARADIGM FOR EXPLOITING FINE-GRAIN PARALLELISM \nAbstract: We propose a new processing paradigm, called the Expandable Split Window (ESW) paradigm, for exploiting fine-grain parallelism. This paradigm considers a window of instructions (possibly having dependencies) as a single unit, and exploits fine-grain parallelism by overlapping the execution of multiple windows. The basic idea is to connect multiple sequential processors, in a decoupled and decentralized manner, to achieve overall multiple issue. This processing paradigm shares a number of properties of the restricted dataflow machines, but was derived from the sequential von Neumann architecture. We also present an implementation of the Expandable Split Window execution model, and preliminary performance results. ",
1105 "neighbors": [
1106 249,
1107 373,
1108 652,
1109 735
1110 ],
1111 "mask": "Test"
1112 },
1113 {
1114 "node_id": 87,
1115 "label": 5,
1116 "text": "Title: A Hybrid Nearest-Neighbor and Nearest-Hyperrectangle Algorithm \nAbstract: We propose a new processing paradigm, called the Expandable Split Window (ESW) paradigm, for exploiting fine-grain parallelism. This paradigm considers a window of instructions (possibly having dependencies) as a single unit, and exploits fine-grain parallelism by overlapping the execution of multiple windows. The basic idea is to connect multiple sequential processors, in a decoupled and decentralized manner, to achieve overall multiple issue. This processing paradigm shares a number of properties of the restricted dataflow machines, but was derived from the sequential von Neumann architecture. We also present an implementation of the Expandable Split Window execution model, and preliminary performance results. ",
1117 "neighbors": [
1118 47,
1119 383,
1120 719,
1121 2021,
1122 2245
1123 ],
1124 "mask": "Train"
1125 },
1126 {
1127 "node_id": 88,
1128 "label": 6,
1129 "text": "Title: Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation \nAbstract: Selecting a good model of a set of input points by cross validation is a computationally intensive process, especially if the number of possible models or the number of training points is high. Techniques such as gradient descent are helpful in searching through the space of models, but problems such as local minima, and more importantly, lack of a distance metric between various models reduce the applicability of these search methods. Hoeffding Races is a technique for finding a good model for the data by quickly discarding bad models, and concentrating the computational effort at differentiating between the better ones. This paper focuses on the special case of leave-one-out cross validation applied to memory-based learning algorithms, but we also argue that it is applicable to any class of model selection problems. ",
1130 "neighbors": [
1131 44,
1132 116,
1133 208,
1134 225,
1135 251,
1136 371,
1137 587,
1138 682,
1139 760,
1140 762
1141 ],
1142 "mask": "Validation"
1143 },
1144 {
1145 "node_id": 89,
1146 "label": 6,
1147 "text": "Title: NP-Completeness of Searches for Smallest Possible Feature Sets a subset of the set of all\nAbstract: In many learning problems, the learning system is presented with values for features that are actually irrelevant to the concept it is trying to learn. The FOCUS algorithm, due to Almuallim and Dietterich, performs an explicit search for the smallest possible input feature set S that permits a consistent mapping from the features in S to the output feature. The FOCUS algorithm can also be seen as an algorithm for learning determinations or functional dependencies, as suggested in [6]. Another algorithm for learning determinations appears in [7]. The FOCUS algorithm has superpolynomial runtime, but Almuallim and Di-etterich leave open the question of tractability of the underlying problem. In this paper, the problem is shown to be NP-complete. We also describe briefly some experiments that demonstrate the benefits of determination learning, and show that finding lowest-cardinality determinations is easier in practice than finding minimal determi Define the MIN-FEATURES problem as follows: given a set X of examples (which are each composed of a a binary value specifying the value of the target feature and a vector of binary values specifying the values of the other features) and a number n, determine whether or not there exists some feature set S such that: We show that MIN-FEATURES is NP-complete by reducing VERTEX-COVER to MIN-FEATURES. 1 The VERTEX-COVER problem may be stated as the question: given a graph G with vertices V and edges E, is there a subset V 0 of V , of size m, such that each edge in E is connected to at least one vertex in V 0 ? We may reduce an instance of VERTEX-COVER to an instance of MIN-FEATURES by mapping each edge in E to an example in X, with one input feature for every vertex in V . 1 In [8], a \"proof\" is reported for this result by reduction to set covering. The proof therefore fails to show NP-completeness. nations.",
1148 "neighbors": [
1149 430,
1150 635,
1151 638
1152 ],
1153 "mask": "Train"
1154 },
1155 {
1156 "node_id": 90,
1157 "label": 2,
1158 "text": "Title: The wake-sleep algorithm for unsupervised neural networks \nAbstract: We describe an unsupervised learning algorithm for a multilayer network of stochastic neurons. Bottom-up recognition connections convert the input into representations in successive hidden layers and top-down generative connections reconstruct the representation in one layer from the representation in the layer above. In the wake phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the sleep phase, neurons are driven by generative connections and recognition connections are adapted to increase the probability that they would produce Supervised learning algorithms for multilayer neural networks face two problems: They require a teacher to specify the desired output of the network and they require some method of communicating error information to all of the connections. The wake-sleep algorithm finesses both these problems. When there is no teaching signal to be matched, some other goal is required to force the hidden units to extract underlying structure. In the wake-sleep algorithm the goal is to learn representations that are economical to describe but allow the input to be reconstructed accurately. Each input vector could be communicated to a receiver by first sending its hidden representation and then sending the difference between the input vector and its top-down reconstruction from the hidden representation. The aim of learning is to minimize the description length which is the total number of bits that would be required to communicate the input vectors in this way [1]. No communication actually takes place, but minimizing the description length that would be required forces the network to learn economical representations that capture the underlying regularities in the data [2]. the correct activity vector in the layer above.",
1159 "neighbors": [
1160 680
1161 ],
1162 "mask": "Validation"
1163 },
1164 {
1165 "node_id": 91,
1166 "label": 2,
1167 "text": "Title: IEEE Learning the Semantic Similarity of Reusable Software Components \nAbstract: Properly structured software libraries are crucial for the success of software reuse. Specifically, the structure of the software library ought to reect the functional similarity of the stored software components in order to facilitate the retrieval process. We propose the application of artificial neural network technology to achieve such a structured library. In more detail, we utilize an artificial neural network adhering to the unsupervised learning paradigm. The distinctive feature of this very model is to make the semantic relationship between the stored software components geographically explicit. Thus, the actual user of the software library gets a notion of the semantic relationship between the components in terms of their geographical closeness. ",
1168 "neighbors": [
1169 745,
1170 747
1171 ],
1172 "mask": "Test"
1173 },
1174 {
1175 "node_id": 92,
1176 "label": 4,
1177 "text": "Title: Learning Analytically and Inductively \nAbstract: Learning is a fundamental component of intelligence, and a key consideration in designing cognitive architectures such as Soar [ Laird et al., 1986 ] . This chapter considers the question of what constitutes an appropriate general-purpose learning mechanism. We are interested in mechanisms that might explain and reproduce the rich variety of learning capabilities of humans, ranging from learning perceptual-motor skills such as how to ride a bicycle, to learning highly cognitive tasks such as how to play chess. Research on learning in fields such as cognitive science, artificial intelligence, neurobiology, and statistics has led to the identification of two distinct classes of learning methods: inductive and analytic. Inductive methods, such as neural network Backpropagation, learn general laws by finding statistical correlations and regularities among a large set of training examples. In contrast, analytical methods, such as Explanation-Based Learning, acquire general laws from many fewer training examples. They rely instead on prior knowledge to analyze individual training examples in detail, then use this analysis to distinguish relevant example features from the irrelevant. The question considered in this chapter is how to best combine inductive and analytical learning in an architecture that seeks to cover the range of learning exhibited by intelligent systems such as humans. We present a specific learning mechanism, Explanation Based Neural Network learning (EBNN), that blends these two types of learning, and present experimental results demonstrating its ability to learn control strategies for a mobile robot using ",
1178 "neighbors": [
1179 136,
1180 479,
1181 552,
1182 565,
1183 1259,
1184 2438
1185 ],
1186 "mask": "Train"
1187 },
1188 {
1189 "node_id": 93,
1190 "label": 3,
1191 "text": "Title: Blocking Gibbs Sampling for Linkage Analysis in Large Pedigrees with Many Loops \nAbstract: Learning is a fundamental component of intelligence, and a key consideration in designing cognitive architectures such as Soar [ Laird et al., 1986 ] . This chapter considers the question of what constitutes an appropriate general-purpose learning mechanism. We are interested in mechanisms that might explain and reproduce the rich variety of learning capabilities of humans, ranging from learning perceptual-motor skills such as how to ride a bicycle, to learning highly cognitive tasks such as how to play chess. Research on learning in fields such as cognitive science, artificial intelligence, neurobiology, and statistics has led to the identification of two distinct classes of learning methods: inductive and analytic. Inductive methods, such as neural network Backpropagation, learn general laws by finding statistical correlations and regularities among a large set of training examples. In contrast, analytical methods, such as Explanation-Based Learning, acquire general laws from many fewer training examples. They rely instead on prior knowledge to analyze individual training examples in detail, then use this analysis to distinguish relevant example features from the irrelevant. The question considered in this chapter is how to best combine inductive and analytical learning in an architecture that seeks to cover the range of learning exhibited by intelligent systems such as humans. We present a specific learning mechanism, Explanation Based Neural Network learning (EBNN), that blends these two types of learning, and present experimental results demonstrating its ability to learn control strategies for a mobile robot using ",
1192 "neighbors": [
1193 725,
1194 759
1195 ],
1196 "mask": "Test"
1197 },
1198 {
1199 "node_id": 94,
1200 "label": 3,