Apache Point Observatory 3.5-meter telescope: operational design and issues

ABSTRACT

The "new-technology"3.5-meter telescope at Apache Point Observatory has been in routine operations since 1994. Designed to enable nearly full remote operation via the Internet, remote use of the telescope comprises two-thirds of all observing. Rapid instrument change capabilities and flexible scheduling allow for some optimized science utilization. Several science programs can share the telescope on a given night, using more than one scientific instrument. Remote users can also collaboratively use the telescope simultaneously from different geographical locations. Synoptic observing programs and rapid-response observations are routinely accommodated. More than two hundred observers have used the telescope remotely, and by the end of 1997 more than 60 scientific publications based on telescope data have appeared in the journals.

Several scenarios for operating the telescope have been explored. The current scheme is to schedule the telescope by quarters based on prioritized proposals submitted by the consortium member institutions. Except for short synoptic observations and targets of opportunity, each night is divided into halves. These half- night blocks provide adequate time on target plus calibration time, and provide simplification of the scheduling process which is done manually.

Enhancements to telescope performance and efficiency are underway, which will provide broader scientific potential and support more exotic scheduling and operational paradigms. "Remote" queue and adaptive scheduling are conceivable, and the use of existing “AI- assisted” scheduling aides is also under consideration.

Keywords: remote observing, Internet, 3.5-meter telescope, Astrophysical Research Consortium, ARC, Apache Point Observatory

1. BACKGROUND

The Astrophysical Research Consortium (ARC) was chartered in 1984 to build and operate observatory facilities at Apache Point for the shared use of consortium astronomers and students. The major projects at the site are the 3.5-meter telescope which has been in operation since 1994, and the Sloan Digital Sky Survey telescope whose installation is near completion. The construction and ongoing operations of the 3.5-meter telescope are funded by the following member institutions: the University of Chicago, New Mexico State University, Princeton University, the University of Washington, and Washington State University, with Johns Hopkins University becoming a member shortly after the telescope was commissioned. Telescope time is allocated to these institutions in proportion to their level of project contributions. Partial construction costs of the telescope were provided by a NSF grant.

Apache Point Observatory (APO) is located in the Sacramento Mountains at 2800 meters elevation near Sunspot, New Mexico. Observing facilities at APO include a 3.5-meter telescope used for general visible and IR imaging and spectroscopy, the SDSS 2.5- meter survey telescope, a smaller telescope which is used to calibrate survey photometry, and a 1-meter telescope owned by NMSU. The site also contains operations and support buildings, plus dormitories. An aerial view of the observatory is shown in Figure 1.

Figure 1. Aerial view of Apache Point Observatory, photograph by Dan Long,

The 3.5-meter telescope and its early operations have been described elsewhere.¹ This paper gives an update on the operational design as it has evolved since the early commissioning, and includes discussion of issues and future enhancements. Although the telescope is still undergoing significant engineering enhancement, the conduct of observing programs on the telescope has become robust and scientifically productive. The inherent flexibility in scheduling and operations&emdash;which are dependent on the hardware design and remote observing capabilities&emdash;has proven its worth. For example, a remote synoptic program which makes quick periodic observations of brightness changes in lensed quasars has derived an independent and robust estimate for H0, the Hubble Constant. Other rapid- response observations have led to the optical identification of a recent gamma-ray burst object.

The telescope exterior and interior views are shown in Figures 2 and 3.

Figure 2. Exterior view of the 3.5-meter telescope, photograph by Dan Long

Figure 3. View of the 3.5-meter telescope from inside enclosure, photograph by Dan Long

The telescope regularly uses an infrared imager and spectrometer, a medium-resolution visible-light spectrograph and imager, and a large-format visible-light CCD imager. A high-speed infrared CCD imager, and a prototype adaptive optics system have also been integrated with the telescope. Also, a number of visitor-supplied instruments have been used with the telescope, some for limited observing runs and others on a continuing basis. An echelle spectrograph is scheduled for delivery at APO this year. Planning for next-generation facility instrumentation has been initiated.

2. OPERATIONAL STRATEGY

The styles in which observatories support users range from "help yourself" to "full-service. " The APO user support model is somewhere in the middle of this range. The users are "owners" of the observatory in a very direct sense; this helps promote a shared and cooperative approach to operations. This operational paradigm is often described as a partnership between astronomers and the observatory, which is facilitated by the remote observing capability and by on-line user information systems. Operational costs are kept down by promoting the direct involvement of the astronomers. Also, a side-benefit of this partnership is ARC astronomers have a first-hand understanding of the data acquisition process and hence are directly involved with the quality of their data. The APO remote observing systems allow this "hands-on" observing approach while enabling flexible and reactive telescope use, and saves significant amounts of astronomers’ time (and funds) by limiting the amount of travel necessary to observe. Since astronomer travel costs are mitigated, new types of observations are made easier such as synoptic observing programs.

The 3.5-meter telescope has a user community estimated to be more than 200 astronomers and students. Principal Investigators must be from ARC-affiliate institutions; collaborators from outside ARC institutions are welcome and numerous. Although not as large a constituency as a national facility, APO serves enough astronomers and students that the user support systems are a similar to some of the practices used at both small private observatories and the larger federally funded institutions.

3. SCIENCE PROGRAM LIFECYCLE

3.1 Proposal support and selection

Each ARC astronomy department selects a faculty "scheduler" and a Users Committee representative. The scheduler conducts the local proposal selection process, and delivers a list of prioritized observing programs to the Director six to seven weeks before the next scheduling quarter. The telescope was originally scheduled on a monthly basis, but a quarterly system has been adopted as more amenable to the users. The Users Committee members facilitates two-way communication between the observatory and the institutional users. User information is also furnished through a web page at APO, or through direct phone or e-mail contact with site staff or other ARC scientists. The APO web home page is shown below in Figure 4, and contains information about the site and visitor logistics as well as technical information for planning observing programs.

Figure 4. APO World Wide Web home page, at www.apo.nmsu.edu

The ARC proposal processing system is relatively simple, inexpensive, and expeditious. The Director issues a call for proposals by e-mail directly to the schedulers at the respective ARC astronomy departments. The call for proposals contains a short section on updated policies, time allocation quotas for each institution, scheduling constraints, and a blank ASCII proposal form. A typical proposal is less than two pages long, partly because only the summary observing parameters need to be specified in the proposal&emdash;the PI or collaborator will generally conduct the program hands-on, so these details and not required in the proposal [Proposal details are available by opening message 254 at http://www.astro.princeton.edu/APO/apo35- general/INDEX.html].

After selection and prioritization, the institutional schedulers send the proposals to the Director electronically, who hand-crafts a three-month schedule. Proposal priorities, lunar and other scheduling constraints, balance between institutional allocations, inclusion of engineering time, etc. are reasonably well accommodated. The majority of the programs are given half-nights unless they require longer or shorter observing duration. Multi- night campaign observing programs as well as half-hour synoptic or target-of-opportunity programs are in the minority but do comprise a significant amount of observing time.

3.2 Observing

Typically, two to three programs are scheduled for separate time intervals in the same night. These science programs often involve different instruments, observers, and institutional affiliations. Observers can either observe at the telescope in person, or remotely through the Internet or modem backup. Multiple remote users can also connect simultaneously from different geographical locations, allowing "eavesdropping" or collaboration at a distance. Routine remote operation of the telescope has been conducted by ARC astronomers from Israel, Canada, the United Kingdom, the Kuiper Airborne Observatory, and once via satellite from the South Pole. Roughly two-thirds of all observing is done remotely, with the remainder being on-site observations or service observing. The observer interface software screen is shown below, Figure 5.

Instrument changes are done at night by a single person, often in less than 5 minutes. The basic design of the telescope will enable these instrument changes to be performed without human assistance in the future, and allow several instruments to be mounted on the telescope simultaneously and kept in "standby mode. "

All telescope operation is supported by an on-duty Observing Specialist, who is responsible for telescope and human safety, as well as providing technological assistance to the on-site or remote astronomer using the telescope. For a small number of programs, the Observing Specialists are active collaborators and conduct the observing.

Visits to the site by astronomers are mainly for installation and testing of new instruments, or for training purposes. Observing functions, which include complete telescope control, instrument control, quick-look quality assurance, and data retrieval, are all accomplished by the remote observer. Remote observing may only be undertaken by, or with the direct help and supervision of, observers with on-site experience and training. Normally, this is taken to be at least 3 nights of time at APO. At the site, some help for experienced observers can be provided by the Observatory staff but training of graduate students or other inexperienced observers is not available; it is the responsibility of ARC institution faculty and staff.

3.3 Post-observation support and user feedback

To help keep operating costs down, data analysis and archive functions are assumed to be provided by the users’ home institutions. IRAF and IDL are available on-site for quick-look inspection, and sufficient disk storage space is provided to allow user data files to be stored at the site for up to a week before automatic erasure.

Nightly user feedback is collected by the Observing Specialist and summarized in the night logs. If technical problems are encountered, a web-based problem reporting systems is in place, which is managed and attended to by the daytime staff. Monthly telephone meetings of the 3.5-meter Users Committee are held during the year and the minutes are electronically published through e-mail distribution, to which users subscribe. There is no newsletter, but a yearly users meeting is held at APO to discuss recent science and engineering progress with the telescope, and to canvass opinion on long -term planning and observatory priorities.

4. SUPPORTING SYSTEMS

4.1 Observatory staff

The on-site APO operations staff are NMSU employees whose time is split evenly between support of the 3.5-meter telescope and the SDSS, with the exception of the Observing Specialists and Electronics Technician who work exclusively on the 3.5-meter. On- and off-site support for the 3.5-meter telescope is provided by slightly more than 10 full-time equivalents (FTEs), consisting of the following job titles and levels-of-effort:

• Site Director (0.1 FTE)
• Site Operations Manager (0.5 FTE)
• Deputy Site Manager, Observatory Engineer (0.5 FTE)
• Telescope Systems Engineer (0.5 FTE)
• Electronics Technician (1.0 FTE)
• Observatory Computer Systems Manager (0.5 FTE)
• Observing Specialists (3.5 FTE)
• Technical Writer (0.5 FTE)
• Data Aid/Housekeeping (0.5 FTE)
• Maintenance (0.5 FTE)
• Accounting Technician (0.45 FTE)
• Clerical Aide (0.22 FTE)

Substantial support is provided by off-site personnel, including:

• Observatory Director (0.1 FTE)
• 3.5-meter Director (0.25 FTE)
• Telescope Scientist (0.3 FTE)
• Operations s/w maintenance (0.1 FTE)
• Instrument Engineering (0.2 FTE)
• ARC administration (0.25 FTE)
• Mechanical Engineering (0.25 FTE)

The site operations staff generally work a 5-day Monday through Friday work shift. For serious technical problems, nighttime and weekend emergency coverage is arranged in an ad hoc fashion. Authorized staffing levels do not support scheduled on-call engineering support. The Observing Specialists who oversee nighttime telescope operations work an innovative 21-day work cycle: 7 nights consecutive, 7 days off, 5 days mid-shift (noon to 8 p.m.), two days off. An issue with this schedule is fatigue in the winter months when the nights are longest. To minimize the effects of fatigue on efficiency and safety, consideration is being given to splitting responsibility for the longest nights between two Observing Specialists.

4.2 Engineering time

About 7% of telescope time is nominally scheduled for routine engineering tasks, which include periodic corrections to pointing models, optical collimation, calibrations, system throughput measurements, operating software upgrades, training, and small repairs and tune-ups that can wait until a scheduled night or half-night of engineering time. Site engineering staff work in conjunction with off-site ARC astronomers and engineers to plan and conduct these activities. The engineering observing is often conducted remotely with the help of an on-site Observing Specialist.

Major engineering shutdowns are planned and executed on a "when ready" basis. These shutdowns have been allocated as much as 25% of available telescope time during a year, the time being used to effect major upgrades and repairs to the telescope. When all the prerequisites are in place to install a major system upgrade (e.g., a new guider) or perform substantial maintenance (e.g., realuminize the primary mirror), the Director pre-empts the planned observing schedule with at least two weeks advance warning. This practice stems from the difficulties of planning large engineering shutdowns months in advance because of manpower limitations and the uncertainties related to various R&D aspects of the work and its preparation.

4.3 Telecommunications

APO has a dedicated T1 circuit to NMSU enabling remote operation of the 3.5-meter telescope through the Internet, nominally at about 10% of T1 data rates due to Internet bottlenecks elsewhere. A modem backup system is in place. Because new instruments with larger CCDs have begun to tax the existing network systems, plans for increasing the data bandwidth to the site are in progress. Many of the ARC institutions already have or have proposed to acquire vBNS or Internet2 network capability.

4.4 Environmental monitoring

Clouds, wind, dust, condensing humidity, and precipitation are routinely monitored because of their effect on data quality and possible risk to the telescope. Real time weather-sensing transducers and a cloud monitor are in place, and their outputs are collected and served through a web page, showing current as well as past conditions. Criteria for telescope closure are posted and enforced unilaterally by the Observing Specialists. The APO weather web page is shown in Figure 6. Figure 7 shows a thermal infra-red all-sky image which is taken by a prototype a 10-micron scanner in routine use at the site.

Another environmental issue is light pollution. In order to preserve the relatively dark skies at APO, observatory management conducts an active program which promotes dark-sky legislation and community education. Outdoor lighting ordinances are in place in the local communities of Alamogordo and Cloudcroft, and legislation or guidelines are in preparation for Otero County, the state of New Mexico, and local federal facilities such as Holloman Air Force Base.

5. UPGRADES AND ENHANCEMENTS

5.1 Performance

Although scientifically capable, our early experience with the telescope has uncovered various deficiencies which prevent users from benefiting from the full performance and efficiency capabilities inherent in the telescope design. Defective enclosure wheels were replaced. The primary mirror pneumatic support system was redesigned and implemented. Improved primary ventilation hardware was installed. A larger, more sensitive guide camera was integrated into the off-axis Nasmyth instrument port. Advanced means of aligning the optics were devised and used to improve image quality. Computer and network systems were maintained and upgraded to give higher capacity and better reliability for operations. Extra funds were earmarked to accomplish a substantial list of improvements to the observatory, including the replacement of the secondary mirror, enabling automatic tertiary mirror rotation and mirror covers, improving instrument capabilities, modernization of operational software, etc.

The resources for major enhancements to the telescope, existing instruments, and the development of new instruments is not wholly included in the operational budgets. However, the current level of site engineering support is much higher than the early operational concepts, partly because ongoing maintenance, improvements, and enhancement projects require a substantial engineering presence at the observatory. Improvements and enhancement work is generally treated as special projects and undertaken by one or more ARC affiliates, working with the site staff and in concert with the priorities of the Director and ARC Board of Governors. Significant input to this process is derived from the community of users, using mechanisms given above.

5.2 Efficiency

As the intrinsic performance of the telescope continues to improve, the efficiency of obtaining observations becomes increasingly important to the users. Planned upgrades which will improve observing efficiency include:

• tertiary mirror automation-enables faster instrument changes during the night;
• autofocus with guider-improves throughput and eliminates need for periodic refocus;
• "AI" and reactive scheduling-better optimization of telescope use to available time and conditions;
• telescope engineering monitoring system-telescope problem diagnostics and efficiency metrics;
• network upgrades-provide better latency and bandwidth for remote observing and large format CCDs;
• queue-remote observing-better matches observing conditions to observing requirements;
• service observing-enables science observing without presence of principal investigators;
• adaptive optics-more flux per pixel on faint objects (needs laser beacon), plus higher resolution; and
• new instruments-greater throughput and science capabilities, lower noise detectors, etc.

5.3 Future directions

As the APO 3.5-meter telescope approaches its second half-decade of operation, the early commissioning problems are largely understood and either fixed or in process of being fixed. The telescope and instruments routinely provide the user community with useful and often exciting data. Looking forward, the ARC community has begun discussion on this question: is the 3.5-meter to become an excellent general-purpose visible/IR telescope, with current technology instruments, or do we concentrate limited resources toward development of a unique scientific capability. Perhaps a balance between these two paths is possible. In the coming decades, telescopes of this size with large-field capabilities, high resolution imaging, state-of-the art CCD imagers and spectrometers, flexible scheduling, remote operations, and easy access for users, will undoubtedly play a significant part toward future advancements in astronomy.

6. ACKNOWLEDGMENTS

My appreciation to Ed Turner and Kurt Anderson for reviewing the manuscript, and to Dan Long for furnishing the photographs and graphics.

7. REFERENCES

1. B. Gillespie, R.F. Loewenstein, and D. York, "Remote Observing at Apache Point", New Observing Modes for the Next Century, ASP Conference Series, 87, pp. 97-108, 1996.