Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.stsci.edu/~wms/ResCulture.doc Дата изменения: Sat Aug 15 22:24:41 2009 Дата индексирования: Mon Oct 1 22:41:11 2012 Кодировка:

(Rio GA---Spec. Session 5)
Discovery and the Culture of Astronomy

Robert Williams
Space Telescope Science Institute

Abstract: The process by which discoveries are made and accepted is
central to progress in astronomy. The manner in which papers are refereed,
telescope time is awarded, and research funds are allocated evolves (at
best) slowly with time, lacking a systematic procedure by which it is
evaluated and modified. It would benefit from greater diversity, e.g.,
witness the great success of astro-ph. The culture by which astronomy is
done should be subjected from time to time to the same scrutiny as research
proposals and journal publications. Decadal surveys, in those countries
that undertake them, should be a good mechanism for conducting such process
assessment and recommending change.

I was asked to start off this session on discovery by making general
remarks that might provide a perspective for subsequent discussion. Let me
begin by stating that the act of astronomical discovery per se is only a
part of what is relevant to this session. In fact, it may be a small part.
Agreeing on what constitutes discovery and assessing the process by which
discovery occurs and is accepted are an integral part of accelerating
discovery. One can point to 'discoveries' such as the Saha equation that
were not initially accepted for publication and which were therefore
delayed or denied. Discoveries have little significance as long as they
remain only with the discoverer. The recent success of astro-ph with its
largely unfiltered acceptance of manuscripts has proven quite effective in
the dissemination of ideas, contrasting as it does with the refereed
journals. This raises interesting questions. Is astro-ph successful
because of the journals? Are there checks and balances that will prevent
it from evolving into electronic babble?, i.e., a diversion akin to
television talk shows? Clearly, the entire process by which research is
done, i.e., the culture of science, is central to discovery. In addition
to fostering creative spark, accelerating discovery involves optimizing the
research process.
Astronomical discoveries are made through observations and their
interpretation, and through theoretical results that follow from
fundamental principles and which are verified by observations. These
require knowledgeable individuals who generally have need for funding.
Consider some of the components involved in this process. First,
education. To take just one example, graduate curricula have in the past
often concentrated on basic physics/astrophysics and observational
techniques to the detriment of instrumentation development. This situation
has been recognized and addressed in recent years and there are now more
graduate programs offering a specialization in instrumentation. This is an
important step in achieving a balance that should be more conducive to
discoveries.
Research infrastructure is one of the most important catalysts for
discovery. New phenomena are inevitably discovered when novel equipment
and techniques are applied to observations. However, creating and
maintaining a forefront infrastructure is costly and it creates the debate
that large, innovative projects divert funds from more traditional
infrastructure development that supports the work of the broader community.
This competition for resources within the community is inevitable and
healthy, and is usually dealt with by the peer review process. On a larger
scale the competition for funds has motivated some countries to undertake
decade reviews in order to set priorities.
Are there criteria that we can agree upon that can be invoked when
making major funding decisions among competing groups? Later today two of
the sessions of this Spec. Session will be devoted to some questions that
some of us have wrestled with (1) concentration of large facilities at
national centers vs. smaller facilities from university departments, and
(2) what is the most effective way to assign time on major telescopes?
Simply on grounds of lack of diversity one could question whether the
current reliance of telescope time allocation on peer review committees is
optimal because so many different observatories have adopted the same
system. In a world in which the e-folding time for change of everything
is of order ten years, it is healthy to be skeptical of any process that
has remained unchanged for more than a generation. Most peer review
processes have remained basically the same for the past half century. It
is important to have a mechanism for assessing and optimizing process by
making modifications. I remain puzzled (read that: frustrated) by the
reluctance of communities to give a greater fraction of telescope time to
the director of a facility to assign, especially when that director reports
to a council and is him/herself reviewed regularly. High risk, high reward
projects require hard decisions. Hard decisions are more readily made by
individuals, not committees. At least 20-25% of time should be awarded by
the director of a facility. Such a policy would definitely accelerate
discovery.
Funding and publication are critical elements in discovery, and both of
them involve peer review. The peer review system has evolved over the
years to its current state. Is it the most effective system that it can be
in supporting discovery? It would be useful to develop criteria by which
the effectiveness of the peer review system can be assessed.
One of the dogmas of science is that novel ideas tend to emanate from
single individuals. Committees have a tendency to minimize risk.
Individuals are less constrained. It is noteworthy that the peer review
system for publication has evolved differently than that for funding and
telescope time assignment. The journal refereeing system relies primarily
on a single individual, the referee, who the editor believes is
knowledgeable about the subject of the manuscript. By contrast, funding
decisions are generally recommended by committees of many individuals
representing a broad range of expertise. There are reasons for this
difference, e.g., the amount of funds available is usually limited whereas
there is no hard limit to the number of journal pages that can be
published. But, it does call attention to the fact that different
circumstances may call for different processes, and these processes are a
sufficiently important part of doing science that they merit analysis and
scrutiny. Better to exercise some control over them than to simply let
them passively evolve.
When possible, metrics should be developed that will aid in the
assessment of process. Consensus may be difficult to achieve on their
validity so they may not be decisive in establishing the best courses of
action in a situation where subjective judgment tends to rule. Therefore,
experimentation with different processes may be more effective and should
be encouraged, as this is likely to be the best way of arriving at good
processes. Such experimentation may, in fact, be an important reason why
the present peer review system and culture in astronomy is so uniform world-
wide. The community should be flexible in accepting different procedures
for similar activities in different organizations.
Many of us are familiar with an environment where every aspect of a
project is subjected to review in a process called program management. All
components of a project are subjected to cost vs. benefit analysis. I
would not advocate this on a continuing basis for process or culture,
however I do think it makes sense to subject important aspects of the
culture of doing science to periodic analysis. It is important that we ask
questions about every facet of how we advance discovery, and then find ways
to make corrections to those processes. Let's subject to occasional
scrutiny such issues as: (a) to what extent should national centers give
telescope time to private groups who provide new instrumentation at their
expense? (b) should the terms of postdocs be limited to 3 years? (c) do
tenure decisions require 10-15 external letters of reference? (d) would
acceptance of journal papers, i.e., evaluation of the referee's report(s)
and author's response(s) be better made by an independent referee rather
than the editor? (e) Under what conditions is the national community
served best by management organizations such as AURA, AUI, USRA, etc. that
have large numbers of members, including international members, as opposed
to smaller, more cohesive university management organizations? We can
disagree on the answers to these questions, but I find it hard to argue
against creating within our astronomy culture forums where these questions
are raised and aired out.
Cultures are very hard to change, which is not a bad thing since their
evolution is normally driven by the community good. The difficulty occurs
when the environment changes faster than the culture. There is good reason
to have a mechanism in place by which change in process can be an integral
part of science culture. At some level serious attention should be paid to
creating a process by which questions are asked of traditional procedures,
and assessments and recommendations made. As one concrete suggestion I
would advocate that those countries that undertake decade surveys should
address the question of astronomy culture and process. The broad community
participation in the decade reviews and the cachet that they have with all
segments of the community---from scientists and administrators, funding
agencies, and government officials---makes them an ideal vehicle for
assessing astronomy culture and recommending change where it might
accelerate the progress of science.