http://arxiv.org/abs/1302.4720

Abstract:

Links: Coming soon.

Abstract:

Sensor network localization (SNL) is the problem of determining the locations of the sensors given sparse and usually noisy inter-communication distances among them. In this work we propose an iterative algorithm named PLACEMENT to solve the SNL problem.
This iterative algorithm requires an initial estimation of the locations and in each iteration, is guaranteed to reduce the cost function. The proposed algorithm is able to take advantage of the good initial estimation of sensor locations making it suitable for localizing moving sensors, and also suitable for the refinement of the results produced by other algorithms. Our algorithm is very scalable. We have
experimented with a variety of sensor networks and have shown that the proposed algorithm outperforms existing algorithms both in terms of speed and accuracy in almost all experiments. Our algorithm can embed 120,000 sensors in less than 20 minutes.

Links: PDF

Abstract:

Device-free localization systems pinpoint and track people in buildings using changes in the signal strength measurements made on wireless devices in the building’s wireless network. It has been shown that such systems can locate people who do not participate in the system by wearing any radio device, even through walls, because of the changes that moving people cause to the static wireless network. However, many such systems cannot locate stationary people. We present and evaluate a system which can locate stationary or moving people, with or without calibration, by quantifying the difference between two histograms of signal strength measurements. From five experiments, we show that our kernel distance-based radio tomographic localization system performs better than the state-of-the-art device-free localization systems in different non line-of-sight environments.

Abstract:

In distributed learning, the goal is to perform a learning task over data distributed across multiple nodes with minimal (expensive) communication. Prior work (Daume III et al., 2012) proposes a general model that bounds the communication required for learning classifiers while allowing for $\eps$ training error on linearly separable data adversarially distributed across nodes.

In this work, we develop key improvements and extensions to this basic model. Our first result is a two-party multiplicative-weight-update based protocol that uses $O(d^2 \log{1/\eps})$ words of communication to classify distributed data in arbitrary dimension $d$, $\eps$-optimally. This readily extends to classification over $k$ nodes with $O(kd^2 \log{1/\eps})$ words of communication. Our proposed protocol is simple to implement and is considerably more efficient than baselines compared, as demonstrated by our empirical results.
In addition, we illustrate general algorithm design paradigms for doing efficient learning over distributed data. We show how to solve fixed-dimensional and high dimensional linear programming efficiently in a distributed setting where constraints may be distributed across nodes. Since many learning problems can be viewed as convex optimization problems where constraints are generated by individual points, this models many typical distributed learning scenarios. Our techniques make use of a novel connection from multipass streaming, as well as adapting the multiplicative-weight-update framework more generally to a distributed setting. As a consequence, our methods extend to the wide range of problems solvable using these techniques.

Abstract:

We consider the problem of learning classifiers for labeled data that has been distributed across several nodes. Our goal is to find a single classifier, with small approximation error, across all datasets while minimizing the communication between nodes. This setting models real-world communication bottlenecks in the processing of massive distributed datasets. We present several very general sampling-based solutions as well as some two-way protocols which have a provable exponential speed-up over any one-way protocol. We focus on core problems for noiseless data distributed across two or more nodes. The techniques we introduce are reminiscent of active learning, but rather than actively probing labels, nodes actively communicate with each other, each node simultaneously learning the important data from another node.

Links: PDF (this is the submitted version, not the final accepted version)

Abstract:
Multidimensional scaling (MDS) is one of the most popular methods for reducing the dimensionality of data. As data sizes have grown, the space and time limitations of traditional MDS algorithms have become more pronounced, and extensive research has gone into designing methods for performing MDS that scale to larger data sets. However, these approaches generally start with a matrix decomposition approach to solving MDS. This matrix decomposition is expensive in time and space, and thus the approaches focus on trying to approximate the decomposition using Nyström methods to solve a smaller matrix decomposition problem.

In this paper, we present a new approach to scalable MDS that combines adaptive sampling methods, multi-pass streaming algorithms, and multi-core extensions, and gives a much better error-time tradeoff than prior approaches. Our approach uses a nonlinear projection technique that was recently developed for MDS and avoids expensive matrix decompositions, from which it derives much of its space and time efficiency.

This method allows us to perform MDS feasibly and accurately on data sets of the order of hundreds of thousands of points. While this is still not “enormous”, it is orders of magnitude larger (for similar error rates) than previous known methods. In addition, because of the underlying approach we use, this method generalizes to many variants of MDS (using robust error metrics, in non-Euclidean spaces) that have never been studied at scale.

Many data analysis problems in machine learning, shape analysis, information theory and even mechanical engineering involve the study and analysis of collections of positive definite matrices. The space of such matrices P(n) is a Riemannian manifold with variable negative curvature. It includes Euclidean space and hyperbolic space as submanifolds, and poses significant challenges for the design of algorithms for data analysis.

In this paper, we develop foundational geometric structures and algorithms for analyzing collections of such matrices. A key technical contribution of this work is the use of \emph{horoballs}, a natural generalization of halfspaces for non-positively curved Riemannian manifolds. Horoballs possess some desirable properties of halfspaces (and balls) but are fundamentally more complex to work with because of the inherent curvature of the underlying space.

We propose generalizations of the notion of a convex hull and a centerpoint and develop algorithms for constructing such structures approximately by combining structural properties of horoballs with novel decompositions of P(n). Using these, we also prove that the VC-dimension of range spaces defined by horoballs is bounded in the case of P(2) (2 x 2 symmetric positive definite matrices).

Links:

• 2 page version at the 19th Fall Workshop on Computational Geometry
• Original version at the arxiv (arXiv:0912.1580v2 [cs.CG])
• Latest version (restructured, including new results on VC-dimension):

Abstract:

Bregman divergences are important distance measures that are used extensively in data-driven applications such as computer vision, text mining, and speech processing, and are a key focus of interest in machine learning. Answering nearest neighbor (NN) queries under these measures is very important in these applications and has been the subject of extensive study, but is problematic because these distance measures lack metric properties like symmetry and the triangle inequality.

In this paper, we present the first provably approximate nearest-neighbor (ANN) algorithms. These process queries in $O(\log n)$ time for Bregman divergences in fixed dimensional spaces. We also obtain $\text{poly}\log n$ bounds for a more abstract class of distance measures (containing Bregman divergences) which satisfy certain structural properties . Both of these bounds apply to both the regular asymmetric Bregman divergences as well as their symmetrized versions.

To do so, we develop two geometric properties vital to our analysis: a reverse triangle inequality (RTI) and a relaxed triangle inequality called $\mu$-defectiveness where $\mu$ is a domain-dependent parameter. Bregman divergences satisfy the RTI but not $\mu$-defectiveness. However, we show that the square root of a Bregman divergence does satisfy $\mu$-defectiveness. This allows us to then utilize both properties in an efficient search data structure that follows the general two-stage paradigm of a ring-tree decomposition followed by a quad tree search used in previous near-neighbor algorithms for Euclidean space and spaces of bounded doubling dimension.

Our first algorithm resolves a query for a $d$-dimensional $(1+\eps)$-ANN in $O \left((\frac{\log n}{\eps})^{O(d)} \right)$ time and $O(\left(n \log^{d-1} n \right)$ space and holds for generic $\mu$-defective distance measures satisfying a RTI. Our second algorithm is more specific in analysis to the Bregman divergences and uses a further structural constant, the maximum ratio of second derivatives over each dimension of our domain ($c_0$). This allows us to locate a $(1+\eps)$-ANN in $O(\log n)$ time and $O(n)$ space, where there is a further $(c_0)^d$ factor in the big-Oh for the query time.

We provide a new framework for generating multiple good quality partitions (clusterings) of a single data set. Our approach decomposes this problem into two components, generating many high-quality partitions, and then grouping these partitions to obtain k representatives. The decomposition makes the approach extremely modular and allows us to optimize various criteria that control the choice of representative partitions.

Links: PDF

Links: PDF