This is a TEST

By John Regehr

Ranking university programs can be useful: it helps students decide which school to attend, it helps prospective professors decide where to apply for jobs, and it lets university administrators determine which of their units are performing exceptionally well.

What does it really mean for one department to be ranked higher than another? Does it mean that they publish more papers? That more of their graduates create successful companies? It isn’t clear that there’s any single right answer to these questions.

It is clear, however, that ranking can be done badly, and unfortunately, according to the Computing Research Association, this is what has happened to the US News and World Report rankings for computer science, which is perhaps the most widely used and influential ranking. The CRA issued a statement describing a number of problems with the methods used by US News and World Report–including the fact that they do a poor job tracking the venues where computer scientists publish research papers–and concludes: “Anyone with knowledge of CS research will see these rankings for what they are—nonsense–and ignore them. But others may be seriously misled.”

Suresh Venkatasubramanian featured on Science Friday

September 17th, 2018

Professor of computing at the University of Utah Suresh Venkatasubramanian talks with Ira Glass and [...]

CRA Statement

Beyond the problems identified by the CRA, the US News rankings are also hard to interpret since the criteria they are based on are not public. We don’t know the formula they use, nor do we have access to the data that they use as input to the secret formula. This makes it hard for people, such as prospective students, to get benefit from rankings, because it just isn’t clear what it means for one computer science department to be ranked above another.

Beyond the problems identified by the CRA, the US News rankings are also hard to interpret since the criteria they are based on are not public. We don’t know the formula they use, nor do we have access to the data that they use as input to the secret formula. This makes it hard for people, such as prospective students, to benefit from rankings.

Computer science professor Emery Berger, at the University of Massachusetts at Amherst, has come up with a better way to do rankings called CSRankings. His method is transparent: anyone can inspect the formula that it uses and also the data is fed into the formula. The entire implementation for his ranking system is available as open source software!

CSRankings is based on the idea that the best computer science departments are the ones that publish the most articles at “top tier” conferences. These conferences might accept only 10-20% of the papers submitted for publication each year and they are where the best researchers tend to submit their best work.

Prospective Students can Explore Ranking Data

Ranking university programs can be useful: it helps students decide which school to attend, it helps prospective professors decide where to apply for jobs, and it lets university administrators determine which of their units are performing exceptionally well. What does it really mean for one department to be ranked higher than another? Does it mean that they publish more papers? That more of their graduates create successful companies? It isn't clear that there's any single right answer to these questions. Read more...

By counting only top-tier publications, instead of total publications, CSRankings avoids the problem of inflating the ranking of researchers who publish a large number of low-quality publications. The CSRankings system is carefully designed to be a zero-sum game: the total credit that it gives to a top-tier paper cannot be inflated by adding authors to a paper.

The openness of the CSRankings system and its data set is a huge advantage. The best thing is that the CSRankings web site allows everyone to explore the data.

Let’s say that a prospective student is interested in operating systems and formal verification. That person can select only those two areas of interest and the site will show the departments that publish heavily in top-tier conferences in those specific areas. A prospective student can then drill down at the department level and see who the key players are in those areas and read their code and papers.

This is a fundamentally different use of a ranking system. The ultimate purpose of the rankings is to guide us toward accurate data that can be used to make informed decisions.

SoC Faculty Receive NSF CAREER Awards

Two University of Utah School of Computing faculty members received the National Science Foundation’s CAREER Award for projects in developing faster cloud-based data systems and software that can help researchers and doctors determine why they choose the medical decisions they make.

Ryan Stutsman

University of Utah School of Computing assistant professor Ryan Stutsman, whose research focuses on “big data” and creating more efficient databases, is receiving $550,000 over five years for a project that rethinks the common approach to cloud-based databases.

Lots of data is stored in the cloud, and when it’s analyzed, that data moves around among the many servers that store and analyze it, he said. “We’re trying to come up with a way to share this massive cloud systems but also safely push their operations to the data itself,” he added. “By running code, we allow these cloud users to pull the data from the databases without moving it around as much.”

That means users could analyze much bigger data sets and produce results more quickly. This could be valuable for services such as Facebook, which deals with massive amounts of data every day in real time. Future technologies, such as autonomous vehicles, also could benefit from this new method. “What we’re most focused on is future networks to move data at high bandwidth with low latencies,” Stutsman said.

Stutsman earned his doctorate in computer science from Stanford University and began at the University of Utah as a faculty member in the summer of 2017. He has performed internships at Facebook and the Lawrence Livermore National Laboratory and postdoctoral research for Microsoft.

“It’s a huge honor,” he said about receiving the NSF CAREER Award. “I’m really excited about the work. In the future people, will have huge amounts of data and interact with it really aggressively, and I think with this project we will push the envelope of that.”

Alexander Lex

University of Utah School of Computing assistant professor Alexander Lex has received $512,000 for developing software that will capture the decision-making process of doctors and other researchers by using algorithms and human-computer interaction methods. Lex, who is also a member of the U’s Scientific Computing and Imaging Institute, conducts research on Interactive data visualization, data analysis methods, visual analytics and data science.

The project is specifically focused on helping doctors with cancer diagnostics and those studying the genetic causes of suicide. Typically, doctors will make a series of medical decisions but that process is not analyzed well, and it’s difficult to reproduce why the doctor made those decisions.

Lex’s research will develop a software system so experts can reproduce their decision-making in order to better justice those choices. “I hope this will give people the tool to better communicate and reproduce the data analysis of what they do,” he said.

“This is really exciting for me, and it’s a great honor,” he said about receiving the CAREER Award.

Lex received his doctorate in computer science from the Graz University of Technology in Austria and was a postdoctoral fellow at Harvard University. He began at the University of Utah in 2015.

The NSF CAREER Award is given out to faculty “who have the potential to serve as academic role models in research and education and to lead advances in the mission of their department or organization.”

Lex and Stutsman are so far the second and third University of Utah College of Engineering faculty members to receive the NSF CAREER Award this year. In January, electrical and computer engineering assistant professor Pierre-Emmanuel Gaillardon received the award for his research on developing transistors that can do more, not just work faster.

To learn more about Lex and Stutsman, click on the video below that profiles their work.

Ivan Sutherland Honored

Ivan Sutherland, a University of Utah computer science luminary who taught the likes of Ed Catmull, Alan Kay and was co-founder of the computer graphics firm, Evans & Sutherland, was honored with the Washington Award from the Western Society of Engineers. The award was handed out Feb. 23 during National Engineers Week at the 2018 Chicagoland Engineering Awards Benefit in Rosemont, Ill.

The Washington Award is given out annually to a professional engineer whose work has advanced the welfare of society. It was established in 1916.

Widely regarded as “The Father of Computer Graphics,” Sutherland is known for his pioneering research in the development of graphics along with colleague David C. Evans, who became the University of Utah’s first chair of the computer science department.

Sutherland earned his bachelor’s in electrical engineering from then Carnegie Institute of Technology (now Carnegie Mellon University), a master’s from the California Institute of Technology and a doctorate from the Massachusetts Institute of Technology. He was a professor of computer science at the U from 1968 to 1974 during the department’s historic period in computer science which generated other well-known legends such as John Warnock, Jim Clark, Nolan Bushnell, as well as Catmull and Kay.

In addition to starting Evans & Sutherland, he also invented the first interactive graphics program with geometric constraints known as Sketchpad as well as a head-mounted display that would be a precursor to virtual reality technology. He became the first recipient of the ACM SIGGRAPH Coons Award for his contributions to computer graphics as well as the ACM Turing Award. He has received countless other awards including the Kyoto Prize, the R&D 100 Award, and the ACM Software System Award. He is also an inductee of the National Inventors Hall of Fame.

He currently lives in Portland where he is a visiting scientist and ACM fellow at Portland State University.

Suresh Venkat Radio Interview

Professor of computing at the University of Utah Suresh Venkatasubramanian talks with Larry Mantle on AirTalk regarding algorithms for pre-trial risk assessment.

Interview from 89.3 KPCC “Air Talk”

The decision of whether to release a defendant on bail and on which conditions is usually left in the hands of judges, but some courtrooms are now turning to risk-assessment AI systems in an effort to make the process less biased.

One commonly used system — Laura and John Arnold Foundation’s Public Safety Assessment — is now used in nearly 38 jurisdictions, including four counties and one city in Arizona, and Santa Cruz County in CA. The system processes data on a defendant based on factors such as their prior convictions, past behavior and age, to create two scores on a scale of 1-6: the likelihood that a defendant will skip out on their court date and the likelihood that they will commit another crime. These scores are one of the many factors that a judge can choose to incorporate into their pre-trial sentencing decision.

Proponents of using AI systems in pre-trial sentencing are hopeful that this will reduce human bias and even replace the cash bail system. But critics are afraid that judges will grow too reliant on these scores. And there are concerns that the system itself may have prejudice baked into it. The argument goes that since these risk-assessment systems rely on data about prior convictions and people of color interact more with the criminal justice system because of pre-existing human bias, they will end up with higher risk scores than white defendants.

We talk with a researcher who is currently running a study on the Arnold Foundation’s Public Safety Assessment scoring system, as well as a professor who studies algorithmic fairness.

Full Interview

U Alum’s Company, Pixar, Honored

Pixar Animation Studios, the legendary computer animation house run by University of Utah computer science alumnus Edwin Catmull that has produced such family favorite films as “Toy Story,” “Finding Nemo” and “Up,” was honored with the 2018 IEEE Corporate Innovation Award from the Institute of Electrical and Electronics Engineers.

The Emeryville, California, movie studio was honored with the award in November for its “long history of pioneering innovations in computer animation and computer graphics,” according to a recent IEEE statement. The award will be handed out during the organization’s annual summit in May.

IEEE announced the honor the same time the animation company, which also produced such hits as “Cars,” “Inside Out” and “Monsters Inc.,” had the No. 1 box-office-grossing film in North America. It’s newest film, “Coco,” about a boy’s musical journey during Mexico’s holiday Dia de Los Muertos (Day of the Dead), earned more than $71 million in the U.S. alone.

In 1986, Catmull, along with Apple icon, Steve Jobs, founded Pixar, creating a studio with one of the best track records of any production house in Hollywood. Pixar first exploded on the Hollywood scene in 1995 with the Academy Award-winning “Toy Story” and has since produced an unprecedented number of box office and critical hits. Pixar’s animated feature films have so far won eight Academy Awards.

In 2006, Pixar merged with Disney, and Catmull remained as president of the company while also becoming president of Disney’s Animation Studios, which created other hit animated films, including “Frozen,” “Tangled,” and “Wreck-It Ralph.” His book, Creativity Inc., gives insight into how Catmull runs his companies and how he best elicits creative thinking from its employees.

Catmull first attended the U in 1963 as a physics student but later took computer science classes as graphics were emerging as a technology. It was during this “Camelot” period in the U’s computer science department that Catmull was paving new ground in computers along with other noted U innovators including Nolan Busnell of Atari, interface designer Alan Kay, and John Warnock, who founded Adobe.