Документ взят из кэша поисковой машины. Адрес оригинального документа : http://hea-www.harvard.edu/AstroStat/Stat310_1415/sa_20150519.pdf Дата изменения: Tue May 19 16:16:45 2015 Дата индексирования: Sun Apr 10 11:39:59 2016 Кодировка: Поисковые слова: m 5

Statistical Issues in the Search for Particle Dark Matter
Sara Algeri1,2 Supervisors: Prof. David van Dyk1 and Prof. Jan Conrad2
1 2

Department of Mathematics, Imp erial College London, London, UK Department of Physics, Stockholm University, Stockholm, Sweden.

CHASC May 19, 2015

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

1 / 44


Contents

1 2 3 4 5 6

Motivation Methods Implementation Results Conclusion Future developments

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

2 / 44


Motivation

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

3 / 44


Motivation

A shortcut on Dark Matter
Dark Matter is the substance postulated in the 30s by Fritz Zwicky to explain the evidence of missing mass in the universe. It is hypothesized to constitute 85% of the total matter in the universe. It has never been observed. We do not know what it is made of. The best candidate are WIMPs. How do we detect WIMPs? LHC We look for discrepancies in terms of momentum and energy. Direct detection We look for a WIMPs-atoms collisions. Indirect detection We look for their decay by-products.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

4 / 44


Motivation

The claim of a discovery
When we look for a new particle (e.g. Higgs boson, quark etc.), we look from the presence of a (5 ) line/bump (the signal of the particle) on top of a background flux (what we know). In the case of the search for Dark Matter, we might have something more complicated than a bump, and we can even have a fake signal (i.e something mimicking WIMPs, but not a background to them) We might have to deal with an entirely new distribution! E.g.: In the indirect detection scenario we want to make sure that the gamma rays we observed are due to Dark Matter and not to a different cosmic source (Pulsars for example). What statistical to ol should we use to do so? Op en question that we aim to address with this talk.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

5 / 44


Methods

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

6 / 44


Methods

Brief review on LRT
Suppose we have f (Ei , µ) 3E and we want to test H0 : µ = 1 vs . Ha : µ = 1
th -2 i

+ µ1

{i =10}

with µ 0

(1)

Problem: I want to know if I have a line of intensity 1 in the 10 Solution: Likelihood Ratio Test (LRT) -2 log
n i =1 f (Ei , 1) n ^ i =1 f (Ei , µMLE H0

energy bin.

)



2 1

(2)

Important We can do this just because some regularity conditions hold. Between these, we have: 1. Under H0 , µ is on the interior of the parameter space. 2. The model is identifiable. 3. The models under H0 and Ha are nested.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 7 / 44


Methods

What if condition 1. falls?
1. The parameter of interest lies on the boundary of the parameter space. E.g: We have f (Ei , µ) 3Ei-2 + µ1{i =10} , µ0 we want to test H0 : µ = 0 Practical problem: If there is a line, we know that it occurs at the 10 check if I have a line there or not? Theoretical/practical solutions: Chernoff, 1954
th

(3)

vs . Ha : µ > 0

energy bin. So, how do I

; Bootstrap.
Statistical Tests for Dark Matter CHASC May 19, 2015 8 / 44

S. Algeri (ICL, SU)


Methods

What if condition 2. falls?
2. There exists a nuisance parameters which is defined just under the alternative model. E.g: We have f (Ei , µ, ) 3Ei-2 + µ1{i =} we want to test H0 : µ = 0 vs . Ha : µ > 0 Practical problem: Do I have a line somewhere? Theoretical solution: Davies, 1987 . Practical solution: Gross and Vitells, 2010§ .
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 9 / 44

(4)


Methods

What if condition 3. falls?
3. The plausible models to be tested are non-nested. E.g:

We have
f (Ei , )

E0 +1 E

and

g (Ei , M )

0.73
E 1.5 M

exp -7.8

E M

(5)

we want to test:
Practical problem:

H0 : f (Ei , ) vs . Ha : g (Ei , M )

Are my particles coming from a power law distributed cosmic source or are they coming from Dark Matter? Theoretical solution: Cox, 1961-1962; Atkinson, 1970; etc. Practical solution: Hopefully, this talk (using
S. Algeri (ICL, SU)

,

and

§

) ; Pilla et al., 2005-2006.
CHASC May 19, 2015 10 / 44

Statistical Tests for Dark Matter


Methods

The problem in statistical terms

Let f (y , ) and g (y , ) such that f g for any and f(y,) are g (y , ) non-nested models.

The goal is to develop a test for the hyp otheses: H0 : f (y , ) is the correct model versus Ha : g (y , ) is the correct model

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

11 / 44


Methods

Formulation of the problem
Consider a comprehensive model which includes f (y , ) and g (y , ) as special cases. We have two possibilities: Multiplicative form k {f (y , )}1- {g (y , )} where k= Additive form (1 - )f (y , ) + g (y , ) We prefer the formulation in (8), so that we do not have to worry about dealing with the normalizing constant k . (8) {f (y , )}1- {g (y , )} dy
-1

(6)

(7)

Thus, considering the model in (8) the test reduces to H0 : = 0 versus Ha : > 0
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015

(9)
12 / 44


Methods

The Likelihood Ratio Statistics
Assume that: lies on the interior of its parametric space to be one-dimensional i.e., = . Notice that for fixed, the model (1 - )f (y , ) + g (y , ) 01 (10)

is identifiable and thus the only remaining problem when testing H0 : = 0 versus Ha : > 0 would be being on the boundary WE CAN USE Chernoff, 1954 i.e.:
d

LRT = -2 log[L(0, 0 , 0) - L( , , )] - - 1 2 + 1 (0) ^ ^^ -2 1 2
n

(11)

With L(·) being the likelihood function of the model in (10), , are the ^^ respective ML estimates for , , whereas 0 is the MLE for under H0 . ^
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 13 / 44


Methods

The Likelihood Ratio Statistics process
p Notice that if is not fixed then ( , , ) - ( , , ). ^^^ 1 So, the LRT statistics is asymptotically 2 2 + 1 (0) distributed for 1 2 fixed.

This means that if we let vary {LRT ( ), B} corresponds asymptotically to a 1 2 + 1 (0) random 21 2 process indexed by . Thus the p-value of our test H0 : = 0 versus Ha : > 0 will correspond to the excursion probability P sup LRT ( ) > c
B

(12)

How do we calculate/approximate this?
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 14 / 44


Methods

Approximation of P sup

B

T ( ) > c

From Davies, 1987 we have that if {T ( ), B} is a 2 process, then as c + P (sup T ( ) > c ) P ( 2 c 2 e2 > c) + 2 2 ( 2 + 1 ) 2
-1 c

U

( )d
L

(13)

Expected # of upcrossings over c

if c + we have an upper bound for P (sup T ( ) > c ). ( ) is complicated use the "empirical" version of (13) proposed in Gross and Vitells, 2010 P (sup T ( ) > c ) P (2 > c ) + E [N (c0 )|H0 ]e
-
c -c0 2

c c0

-1 2

(14)

where c0 << c , E [N (c0 )|H0 ] is the number of upcrossings over c0 under the null model (to be estimated via Monte Carlo simulation).
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 15 / 44


Methods

Approximation of P sup

B

LRT ( ) > c

1 So, how do we adjust such results for the case of a 2 2 + 1 (0) 1 2 random process?

We have that P (sup LRT ( ) > c )
pDavies 2



pGV 2 GV

(15) is the

where pDavies is the approximation in (13) with = 1 whereas p correspective empirical version in (14). This holds because of the following result.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

16 / 44


Methods

Approximation of P sup
Result 1.

B

LRT ( ) > c

Let {Yt , t T} be a stochastic process such that t T, Yt = with Wt we have
2 ( )

Wt , if E occurs Vt , if E c occurs
+

, Vt (0) and P (E ) = P (E c ) = 0.5. Then, for c R P (sup{Yt } > c ) pDavies 2

Note: In the case of the our LRT process indexed by , the event E ^ corresponds to d log L( ,, ) =0 < 0, and thus for n we have that d P
d log L( ,, ) ^ d =0
S. Algeri (ICL, SU)

< 0 = 1. 2
Statistical Tests for Dark Matter CHASC May 19, 2015 17 / 44


Methods

Approximation of P sup
Proof of Result 1.

B

LRT ( ) > c

P (sup{Yt } > c ) = P (sup{Yt } > c |E )P (E ) + P (sup{Yt } > c |E c )P (E c ) because of total probabilities 1 1 = P (sup{Wt } > c ) + P (sup{Vt } > c ) 2 2 +) because c > 0 always (c R 1 = P (sup{Wt } > c ) 2 pDavies 2 because Wt 2 ) and c is large. ( Note: All we need is the law of total probabilities!
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 18 / 44


Methods

Alternative method, Pilla et al. 2005-2006
The model of reference is again (1 - )f (y , ) + g (y , ) 01 (16)

we want to test H0 : = 0 versus Ha : > 0. Now, we focus on the normalized Score function S ( ) =
S ( ) nC ( , )

The sup of this will be our test statistics

(17)

and the associated Score process {S ( ) B}, where
n

S ( ) =
i =1

g (yi , ) -1 f (yi , )

(18)

is the Score function of the model in (16) under H0 and C ( , ) = g (yi , )g (yi , ) dyi - 1 f (yi , )
CHASC May 19, 2015

(19)

is the respective covariance function.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter 19 / 44


Methods

Approximation of P sup

B

S ( ) > c , known (Pilla et al. 2005)

The p-value of the test is of the form P (sup S ( ) > c ) Pilla et al. show that for n +, P (sup S ( ) > c ) P (sup Z ( ) > c ) where Z ( ) is a mean zero Gaussian process. Using tube formulae they show that for c (20)

P (sup Z ( ) > c )

Ad

0 +1

P (

2 d +1

c 2 )+ (21) P (2 d
+1-k

d k =1 Ak Ad

k +1-k

c 2)

In words: Ratio between the volume of the tube of radius r (function of c ) built around the manifold associated to sup Z ( ) on a unit sphere, and the volume of the unit sphere itself.

where d is the dimension of , j with j = 0, . . . , d are the geometric 2 w /2 constants depending on specific model to be tested and Aw = ( /2) . w
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 20 / 44


Methods

Approximation of P sup

B

S ( ) > c , unknown (Pilla et al. 2006)

If the nuisance parameter under the null is unknown, the covariance function of the Score process becomes C ( , ) = C ( , ) - C ( |) I-1 ()C ( |)
This is essentially what is changing!

(22)

where is required to lie on the interior of the parameter space, I-1 () is the inverse of the Fisher information matrix whereas C ( |) = g (yi , ) log f (yi , )dy .

is unknown, in the application: C ( , ) can be consistely estimated by C ( , )|= . ^
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 21 / 44


Implementation

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

22 / 44


Implementation

Motivating Examples
General Problem We have n of incoming particles in our detector. We want to know if these n particles come from Dark Matter or from a different cosmic source. We assume that the n particles are distributed as a Marked Poisson random process with marks corresponding to the energies, i.e, n Pois (n, ) LIkelihood:
n

Ei h ( Ei , )

i = 1, . . . , n;

(23)

L() = Pois (n, )
i =1

h(Ei , )

n i =1

h ( Ei , )

(24)

WE CAN FOCUS JUST ON THE DISTRIBUTION OF THE MARKS!
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 23 / 44


Implementation

Example 1: Power law vs. Dark Matter
In this example, the goal is to distinguish the Dark Matter signal from a power law distributed cosmic source. The models for the marks are Power law (Pareto Type I)
E0 E +1

(25)

Dark Matter (from BergstrЁ et al., 1998) om 0.73 E (M ) M with (M ) =
-1.5

exp -7.8
-1.5

E M E dE M

(26)

0.73 E (M ) M

exp -7.8

where E E0 , E0 > 0, M E0 and > 0. In our specific case: E0 = 1, E , M [1; 100].
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 24 / 44


Implementation

Example 2: Power law + power law vs. power law + Dark Matter.
We consider a generalized version of Example 1 which includes a background source also distributed as a power law. The signal can be either power law or Dark Matter. The models for the marks become Power law + power law (1 - ) Power law + Dark Matter (1 - )
E0 0.73 E + +1 E (M ) M -1.5 E0 E0 + +1 E +1 E

(27)

exp -7.8

E M

(28)

where E E0 , E0 > 0, M E0 , , > 0 and 0 < , < 1. Also in this case E is chosen equal to 1, E [1; 100] and M [1; 100].

0

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

25 / 44


Implementation

Example 3: Pulsar Spectrum vs. Dark Matter.
The aim of considering such example is to extend to the statistical framework the difficulty of distinguishing between Dark Matter and pulsar origins discussed in Baltz, 2007. The models in analysis are Pulsar Spectrum (from Baltz, 2007) E E Dark Matter 0.73 E (M ) M where E > E0 , M E0 and , > 0. E0 is chosen to be equal to 0.1, E [0.1; 5], M [0.1; 5] and will be fixed to 2 to guarantee the two models to be non-nested.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 26 / 44

-

exp{-( exp{-(

-

E E0 ) } E E0 ) }dE

(29)

-1.5

exp -7.8

E M

(30)


Implementation

Specify the comprehensive additive model
Let be the parameter of interest to be tested and let M be the nuisance parameter defined just under the alternative model. Then we have: Power law vs. Dark Matter (1 - )
E0 0.73 E + E +1 (M ) M -1.5

exp -7.8

E M

where E0 = 1, M , E [1; 100], 0 1. is unknown. Power law + power law vs. p ower law + Dark Matter (1 - ) (1 - )
E0 E0 E0 0.73 E + +1 + (1 - ) +1 + +1 E E E (M ) M -1.5

exp -7.8

E M

where E0 = 1, M , E [1; 100], 0 1, = = 0.2, = 2. is unknown. Pulsar Sp ectrum vs. Dark Matter (1 - ) E E
-

exp{-( exp{-(

-

E )} E0 E ) }dE E0

+

0.73 E (M ) M

-1.5

exp -7.8

E M

where E0 = 0.1, M , E [0.1; 5], 0 1, = 4/3, = 2.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 27 / 44


Implementation

Verify that Chernoff, 1954 applies

We have seen above that if M is fixed 1 d1 LRT - 2 + (0) 1 2 2 when n +

How big must n be to guarantee that this result holds? Simulate from some values for LRT and from 1 2 + 1 (0). Compare 21 2 them using:
Wilcoxon rank sum test qq-plots histograms empirical cdf graphs etc.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

28 / 44


Implementation
1.0 1.0 0.4 0.6 0.8

Empirical cdf

q

0.0

0

5

10

15

0.0

Wilcoxon test p.value~7e-194

Empirical cdf

0.4

0.6

0.8

Wilcoxon test p.value~0.3111
0 5 10 15

0.2

LRT Statistics
0.30 0.8
q

0.2

LRT Statistics

Density

Density
0 5 10 15

0.20

0.10

0.00

0.0 0

0.2

0.4

0.6

5

10

15

LRT Statistics LRT Statistics
q q q

LRT Statistics LRT Statistics
q

14

q

qq q q q qq qq q q qq qq qq qq q q q q q q q qq qq qq qq qq qq qq q qq qq q qq qq qq qq q q qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qqq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq q q q q

10

q q

qq qq qq q q q q q q q qq qq qq qq qq qq qq qq q qq qq qq q qq qq qq qq qq qq qq qq qq qq q q qq qq qq qq qq qq qq qq qq qq qq qq qq q q q q q qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq qq

qq q q q qq qq q q q

5

0

0

2

0.521 + 0.5(0

4

6

8

10

)

12

14

0

2

4

6

8 10

q q

q

q

q

0

2

0.521 + 0.5(0

4

6

8

10

)

12

14

Figure 1: The left panels refer to the the original simulated values for the LRT statistics. The right panels refer to the adjusted LRT statistics obtained imputing the negative values to b e equal to 0 in order to correct for the effect of floating p oints.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 29 / 44


Implementation

Compute the approximate p-values

LRT-based method
P (sup LRT (M ) > c ) P (2 > c ) 1 + E [N (c0 )|H0 ] e 2
we need this -
c -c0 2

c c0

-1 2

(31)

Pilla et al., 2005-2006
we need this

P (sup Z (M ) > c )



0

2

1 P (2 c 2 ) + P (2 c 2 ) 2 1 2

(32)

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

30 / 44


Implementation

Computing E [N (c0 )|H0 ]
Model Power law vs. Dark Matter Power law + power law vs. power law + Dark Matter Pulsar Spectrum vs Dark Matter Fixed parameters E0 = 1 E0 = 1 = = 0.2 =2 E0 = 0.1 = 4 3 =2 Unknown parameters , M , M M


Sample size 5000 5000 1000

E [N (c0 )|H0 ] c0 = 0.1 0.906 0.867 0.219





Table 1: Estimated numb er of upcrossings of the pro cess LRT (M ) assuming the null model to b e true (i.e., E [N (c0 )|H0 ] ). For all the models 1000 Monte Carlo simulations has been generated. A grid of resolution 100 for the parameter M over the range [1; 100] has been considered for the first two mo dels; whereas a grid of size 20 over the range [0.1;5] was selected for the model in our third example. For all the three cases, the threshold c0 has b een set to 0.1. 1 1 The sample size has b een chosen large enough to guaranteed the 2 2 + 2 (0) for fixed values of 1 M as discussed in the previous slide.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

31 / 44


Implementation

Formulae for 0
If the null model is completely specified
C (M , M ) d 2 C (M ,M ) dM dM dC (M ,M ) dC (M ,M ) dM dM M =M M , M )

-

0 =
M

C(

dM

(33)

If it is not
^ 0 =
M d 2 (M , M )

dM dM



M =M ,= ^

dM

(34)

with (M , M ) being
(M , M ) =
C (M , M ) C (M , M )C (M , M )

.

They look pretty complicated! But notice that to compute them we only need the covariance function of the Score process and a good numerical algorithm to solve the integrals.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 32 / 44


Implementation

Computing 0

Mo del Power law vs. Dark Matter Power law + power law vs. power law + Dark Matter Pulsar Spectrum vs Dark Matter

Fixed parameters E0 = 1 E0 = 1 = = 0.2 =2 E0 = 0.1 = 4 3 =2

Unknown parameters , M


0

5.3379

, M



5.3635

M



2.7397

Table 2: Geometric constants 0 . When the null model ( = 0) is fully specified, 0 is calculated according to equation (33). When a nuisance parameter is present under the null model, its estimate is provided via MLE and 0 is calculated according to equation (34).

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

33 / 44


Results

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

34 / 44


Results

Power law vs. Dark Matter
LRT-based method
1 Sample Size: 10000 MC simulations: 10000 Grid resolution: 100 1

Pilla et al.
Sample Size: 50000 MC simulations: 50000 Grid resolution: 200

0.1

(p.values)

log 3-

0.001

0

5

c

10

15

0.001 0

3-

log10(p.values 0.01

10

0.01

)

0.1

1

2

c

3

4

Figure 2: Blue curves: Approximation for P (sup LRT (M ) > c ) (left panel) and P (sup S (M ) > c ) (right panel). Gray dotted curve: Monte Carlo p-values. Gray area: Monte Carlo errors.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 35 / 44


Results

Power law + power law vs. Power law + Dark Matter
LRT-based method
1 Sample Size: 10000 MC simulations: 10000 Grid resolution: 100 1

Pilla et al.
Sample Size: 50000 MC simulations: 50000 Grid resolution: 200

0.1

(p.values)

log 3-

0.001

0

5

c

10

15

0.001 0

3-

log10(p.values 0.01

10

0.01

)

0.1

1

2

c

3

4

Figure 3: Blue curves: Approximation for P (sup LRT (M ) > c ) (left panel) and P (sup S (M ) > c ) (right panel). Gray dotted curve: Monte Carlo p-values. Gray area: Monte Carlo errors.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 36 / 44


Results

Pulsar Spectrum vs. Dark Matter
LRT-based method
1 1 Sample Size: 1000 MC simulations: 50000 Grid resolution: 40 0.1

Pilla et al.
Sample Size: 100000 MC simulations: 10000 Grid resolution: 1000

(p.values)

0.1

0.001

0

5

c

10

15

1e-04 0

log10(p.values 0.001 3 - 0.01

log 3-

10

0.01

)

1

2

3 c

4

5

6

Figure 4: Blue curves: Approximation for P (sup LRT (M ) > c ) (left panel) and P (sup S (M ) > c ) (right panel). Gray dotted curve: Monte Carlo p-values. Gray area: Monte Carlo errors.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 37 / 44


Results

Power/Type I error comparison

1 0.9 0.8 0.7 0.6 1- 0.5
q N=10, PL

0.1
q N=10, PL

0.09 0.08 0.07

Type I error

N=100, PL N=200, PL N=500, PL N=1000, PL q N=10, DC N=100, DC N=200, DC N=500, DC N=1000, DC

0.06 0.05 0.04 0.03 0.02 0.01

q q q q q q

0.4 0.3 0.2 0.1 0
q q q q q q q

N=100, PL N=200, PL N=500, PL q N=10, DC N=100, DC N=200, DC N=500, DC

q

q

q q

q

q

q q

q q

0 15 30 45 60 75 90

15

30

45 M

60

75

90

M



Figure 5: Power function at 3 (left panel) and Typ e I error at 2 (right panel) at different values of M for the model p ower law vs. Dark Matter.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

38 / 44


Conclusions

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

39 / 44


Conclusions

Which method is better when?
We identified two methods to pursue a test for non-nested mo dels. Which one is better when? Pros LRT-based metho d -If is unknown the theory does not change. -We can do intermediate checks . -The theory is fairly simple. -It app ears more powerful. -Lower Type I error. -It requires smaller n to reach the asymptotic. -It requires Monte Carlo simulations. -It works for Pulsar Sp ectrum vs. Dark Matter. -If c small, we still have an upp er b ound. Cons Pilla et al. 2005-2006 -If is unknown the theory changes. -Intermediate checks cannot be done easily. -The theory is quite complicated. -It app ears less powerful. -Higher Type I error. -It requires larger n to reach the asymptotic. -It requires numerical integrations. -It does not work for Pulsar Spectrum vs. Dark Matter.

Cons LRT-based metho d -It cannot b e applied (yet) if is multidimensional. -To simulate from the null mo del might not b e easy.

Pros Pilla et al. 2005-2006 - if is multidimensional the theory does not change. -Numerical integrations might be simpler.

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

40 / 44


Future developments

Outline

1

Motivation Methods Implementation Results Conclusions Future developments

2

3

4

5

6

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

41 / 44


Future developments

Future developments
Refinements of the LRT-based method.
Loosen the assumption on being one-dimensional. Loosen the assumption on being in the interior of the parameter space.

Evaluate the effect of the resolution of the grid for the nuisance parameter under the alternative model (i.e., M in our three examples). Apply both the LRT-based and the Score-based methods to real data taking in account the measurement of the error. Identify a Bayesian solution for testing non-nested models and compare it the LRT-based approach proposed. Build an R package to implement the procedures presented in this talk.
S. Algeri (ICL, SU) Statistical Tests for Dark Matter CHASC May 19, 2015 42 / 44


References
A. C. Atkinson. "A Metho d For Discriminating Between Mo dels". In: Journal of the Royal Statistical So ciety. Series B (Metho dological) 32.3 (1970). E. A. Baltz, J. E. Taylor, and L. L. Wai. "Can Astrophysical Gamma-Ray Sources Mimic Dark Matter Annihilation in Galactic Satellites?". In: The Astrophysical Journal Letters 659.2 (2007). L. BergstrЁ p. Ullio, and J.H. Buckley. "Observability of rays from dark matter neutralino om, annihilations in the Milky Way halo". In: Astroparticle Physics 9.2 (1998). H. Chernoff. "On the Distribution of the Likeliho od Ratio". In: The Annals of Mathematical Statistics 25.3 (1954). D. R. Cox. "Tests of Separate Families of Hyp otheses". In: Proceedings of the Fourth Berkeley Symp osium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. (1961). D. R. Cox. "Further Results on Tests of Separate Families of Hyp otheses". In: Journal of the Royal Statistical So ciety. Series B (Metho dological) 24.2 (1962). R. B. Davies. "Hypothesis Testing when a Nuisance Parameter is Present Only Under the Alternatives". In: Biometrika 74.1 (1987). E. Gross and O. Vitells. "Trial factors for the look elsewhere effect in high energy physics". In: The European Physical Journal C 70.1-2 (2010), pp. 525-530. R.S. Pilla and C. Loader. "Inference in Perturbation Mo dels, Finite Mixtures and Scan Statistics: The Volume-of-Tube Formula". In: ArXiv Mathematics e-prints (Nov. 2005). R. S. Pilla, C. Loader, and C.C. Taylor. "New Technique for Finding Needles in Haystacks: Geometric Approach to Distinguishing between a New Source and Random Fluctuations". In: Phys. Rev. Lett. 95 (23) (2006).

S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

43 / 44


S. Algeri (ICL, SU)

Statistical Tests for Dark Matter

CHASC May 19, 2015

44 / 44