Question: Integrating a multidimensional integral in scipy Motivation: I have a multidimensional integral, which for completeness I have reproduced below. It comes from the computation of the second virial coefficient when there is significant anisotropy: Here W is a function of all the variables. It is a known function, one which I can define a python function for. Programming Question: How do I get scipy to integrate this expression? I was thinking of chaining two triple quads (scipy.integrate.tplquad) together, but I'm worried about performance and accuracy. Is there a higher dimensional integrator in scipy, one that can handle an arbitrary number of nested integrals? If not, what is the best way to do this?

Response:
With a higher-dimensional integral like this, monte carlo methods are often a useful technique - they converge on the answer as the inverse square root of the number of function evaluations, which is better for higher dimension then you'll generally get out of even fairly sophisticated adaptive methods (unless you know something very specific about your integrand - symmetries that can be exploited, etc.) The mcint package performs a monte carlo integration: running with a non-trivial W that is nonetheless integrable so we know the answer we get (note that I've truncated r to be from [0,1); you'll have to do some sort of log transform or something to get that semi-unbounded domain into something tractable for most numerical integrators):
```
import mcint
import random
import math

def w(r, theta, phi, alpha, beta, gamma):
    return(-math.log(theta * beta))

def integrand(x):
    r     = x[0]
    theta = x[1]
    alpha = x[2]
    beta  = x[3]
    gamma = x[4]
    phi   = x[5]

    k = 1.
    T = 1.
    ww = w(r, theta, phi, alpha, beta, gamma)
    return (math.exp(-ww/(k*T)) - 1.)*r*r*math.sin(beta)*math.sin(theta)

def sampler():
    while True:
        r     = random.uniform(0.,1.)
        theta = random.uniform(0.,2.*math.pi)
        alpha = random.uniform(0.,2.*math.pi)
        beta  = random.uniform(0.,2.*math.pi)
        gamma = random.uniform(0.,2.*math.pi)
        phi   = random.uniform(0.,math.pi)
        yield (r, theta, alpha, beta, gamma, phi)

domainsize = math.pow(2*math.pi,4)*math.pi*1
expected = 16*math.pow(math.pi,5)/3.

for nmc in [1000, 10000, 100000, 1000000, 10000000, 100000000]:
    random.seed(1)
    result, error = mcint.integrate(integrand, sampler(), measure=domainsize, n=nmc)
    diff = abs(result - expected)

    print "Using n = ", nmc
    print "Result = ", result, "estimated error = ", error
    print "Known result = ", expected, " error = ", diff, " = ", 100.*diff/expected, "%"
    print " "
```
Running gives
```
Using n =  1000
Result =  1654.19633236 estimated error =  399.360391622
Known result =  1632.10498552  error =  22.0913468345  =  1.35354937522 %

Using n =  10000
Result =  1634.88583778 estimated error =  128.824988953
Known result =  1632.10498552  error =  2.78085225405  =  0.170384397984 %

Using n =  100000
Result =  1646.72936 estimated error =  41.3384733174
Known result =  1632.10498552  error =  14.6243744747  =  0.8960437352 %

Using n =  1000000
Result =  1640.67189792 estimated error =  13.0282663003
Known result =  1632.10498552  error =  8.56691239895  =  0.524899591322 %

Using n =  10000000
Result =  1635.52135088 estimated error =  4.12131562436
Known result =  1632.10498552  error =  3.41636536248  =  0.209322647304 %

Using n =  100000000
Result =  1631.5982799 estimated error =  1.30214644297
Known result =  1632.10498552  error =  0.506705620147  =  0.0310461413109 %
```
You could greatly speed this up by vectorizing the random number generation, etc. Of course, you can chain the triple integrals as you suggest:
```
import numpy
import scipy.integrate
import math

def w(r, theta, phi, alpha, beta, gamma):
    return(-math.log(theta * beta))

def integrand(phi, alpha, gamma, r, theta, beta):
    ww = w(r, theta, phi, alpha, beta, gamma)
    k = 1.
    T = 1.
    return (math.exp(-ww/(k*T)) - 1.)*r*r*math.sin(beta)*math.sin(theta)

# limits of integration

def zero(x, y=0):
    return 0.

def one(x, y=0):
    return 1.

def pi(x, y=0):
    return math.pi

def twopi(x, y=0):
    return 2.*math.pi

# integrate over phi [0, Pi), alpha [0, 2 Pi), gamma [0, 2 Pi)
def secondIntegrals(r, theta, beta):
    res, err = scipy.integrate.tplquad(integrand, 0., 2.*math.pi, zero, twopi, zero, pi, args=(r, theta, beta))
    return res

# integrate over r [0, 1), beta [0, 2 Pi), theta [0, 2 Pi)
def integral():
    return scipy.integrate.tplquad(secondIntegrals, 0., 2.*math.pi, zero, twopi, zero, one)

expected = 16*math.pow(math.pi,5)/3.
result, err = integral()
diff = abs(result - expected)

print "Result = ", result, " estimated error = ", err
print "Known result = ", expected, " error = ", diff, " = ", 100.*diff/expected, "%"
```
which is slow but gives very good results for this simple case. Which is better is going to come down to how complicated your W is and what your accuracy requirements are. Simple (fast to evaluate) W with high accuracy will push you to this sort of method; complicated (slow to evaluate) W with moderate accuracy requirements will push you towards MC techniques.
```
Result =  1632.10498552  estimated error =  3.59054059995e-11
Known result =  1632.10498552  error =  4.54747350886e-13  =  2.7862628625e-14 %
```


Question:
matplotlib: limits when using plot and imshow in same axes I've been trying to plot an ellipse into an imshow plot. It works, but plotting the ellipse after plotting the image seems to increase xlim and ylim, resulting in a border, which I'd like to get rid of: Note that there is NO white border directly after calling imshow only. My code looks as follows:
```
self.dpi = 100
self.fig = Figure((6.0, 6.0), dpi=self.dpi)
self.canvas = FigureCanvas(self.fig)
self.canvas.setMinimumSize(800, 400)
self.cax = None
self.axes = self.fig.add_subplot(111)
self.axes.imshow(channel1, interpolation="nearest")
self.canvas.draw()
self.axes.plot(dat[0], dat[1], "b-")
```
I've tried setting the limits before and after calling "plot", with no effect
```
# get limits after calling imshow
xlim, ylim = pylab.xlim(), pylab.ylim()
...
# set limits before/after calling plot
self.axes.set_xlim(xlim)
self.axes.set_ylim(ylim)
```
How can I force plot not to increase existing figure limits? Solution (thanks to Joe):
```
#for newer matplotlib versions
self.axes.imshow(channel1, interpolation="nearest")
self.axes.autoscale(False)
self.axes.plot(dat[0], dat[1], "b-")

#for older matplotlib versions (worked for me using 0.99.1.1)
self.axes.imshow(channel1, interpolation="nearest")
self.axes.plot(dat[0], dat[1], "b-", scalex=False, scaley=False)
```

Response:
What's happening is that the axis is autoscaling to match the extents of each item you plot. Images are autoscaled much tighter than lines, etc (imshow basically calls ax.axis('image')). Getting the axis limits before and setting them after should have worked. (It's cleaner to just do limits = axes.axis() before and axes.axis(limits) after, though.) However, if you don't want things to autoscale, it's best to just turn autoscaling off after the initial plot. Try axes.autoscale(False) after plotting the image. As an example, compare this:
```
import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots()
ax.imshow(np.random.random((10,10)))
ax.plot(range(11))
plt.show()
```
With this:
```
import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots()
ax.imshow(np.random.random((10,10)))
ax.autoscale(False)
ax.plot(range(11))
plt.show()
```


Question:
what does numpy.apply_along_axis perform exactly? I have come across the numpy.apply_along_axis function in some code. And I don't understand the documentation about it. This is an example of the documentation:
```

```python
def new_func(a):
    """Divide elements of a by 2."""
    return a * 0.5
```

```python
b = np.array([[1,2,3], [4,5,6], [7,8,9]])
```

```python
np.apply_along_axis(new_func, 0, b)
#Output
#array([[ 0.5,  1. ,  1.5],
#       [ 2. ,  2.5,  3. ],
#       [ 3.5,  4. ,  4.5]])
#```
#As far I as thought I understood the documentation, I would have expected:
#```
#array([[ 0.5,  1. ,  1.5],
#       [ 4  ,  5  ,  6  ],
#       [ 7  ,  8  ,  9  ]])
#```
#i.e. having applied the function along the axis [1,2,3] which is axis 0 in [[1,2,3], [4,5,6], [7,8,9]] Obviously I am wrong. Could you correct me ?
```

Response:
apply_along_axis applies the supplied function along 1D slices of the input array, with the slices taken along the axis you specify. So in your example, new_func is applied over each slice of the array along the first axis. It becomes clearer if you use a vector valued function, rather than a scalar, like this:
```

```python
b = np.array([[1,2,3], [4,5,6], [7,8,9]])
```

```python
np.apply_along_axis(np.diff,0,b)
#Output
#array([[3, 3, 3],
#       [3, 3, 3]])
```

```python
np.apply_along_axis(np.diff,1,b)
#Output
#array([[1, 1],
#       [1, 1],
#       [1, 1]])
#```
#Here, numpy.diff (i.e. the arithmetic difference of adjacent array elements) is applied along each slice of either the first or second axis (dimension) of the input array.
```


Question:
Scipy.optimize Inequality Constraint - Which side of the inequality is considered? I am using the scipy.optimize module to find optimal input weights that would minimize my output. From the examples I've seen, we define the constraint with a one-sided equation; then we create a variable that's of the type 'inequality'. My question is how does the optimization package know whether the sum of the variables in my constraint need to be smaller than 1 or larger than 1? ...
```
def constraint1(x):
    return x[0]+x[1]+x[2]+x[3]-1
```
....
```
con1 = {'type': 'ineq', 'fun': constraint1}
```
link to full solution I'm using in my example: http://apmonitor.com/che263/index.php/Main/PythonOptimization

Response:
Refer to https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/optimize.html and scroll down to Constrained minimization of multivariate scalar functions (minimize), you can find that This algorithm allows to deal with constrained minimization problems of the form: where the inequalities are of the form C_j(x) >= 0. So when you define the constraint as
```
def constraint1(x):
    return x[0]+x[1]+x[2]+x[3]-1
```
and specify the type of the constraint as
```
con1 = {'type': 'ineq', 'fun': constraint1}
```
it automatically assumes that the constraint is in the standard form x[0]+x[1]+x[2]+x[3]-1>=0 i.e., x[0]+x[1]+x[2]+x[3]>=1


Question:
Second Derivative in Python - scipy/numpy/pandas I'm trying to take a second derivative in python with two numpy arrays of data. For example, the arrays in question look like this:
```
import numpy as np

x = np.array([ 120. ,  121.5,  122. ,  122.5,  123. ,  123.5,  124. ,  124.5,
        125. ,  125.5,  126. ,  126.5,  127. ,  127.5,  128. ,  128.5,
        129. ,  129.5,  130. ,  130.5,  131. ,  131.5,  132. ,  132.5,
        133. ,  133.5,  134. ,  134.5,  135. ,  135.5,  136. ,  136.5,
        137. ,  137.5,  138. ,  138.5,  139. ,  139.5,  140. ,  140.5,
        141. ,  141.5,  142. ,  142.5,  143. ,  143.5,  144. ,  144.5,
        145. ,  145.5,  146. ,  146.5,  147. ])

y = np.array([  1.25750000e+01,   1.10750000e+01,   1.05750000e+01,
         1.00750000e+01,   9.57500000e+00,   9.07500000e+00,
         8.57500000e+00,   8.07500000e+00,   7.57500000e+00,
         7.07500000e+00,   6.57500000e+00,   6.07500000e+00,
         5.57500000e+00,   5.07500000e+00,   4.57500000e+00,
         4.07500000e+00,   3.57500000e+00,   3.07500000e+00,
         2.60500000e+00,   2.14500000e+00,   1.71000000e+00,
         1.30500000e+00,   9.55000000e-01,   6.65000000e-01,
         4.35000000e-01,   2.70000000e-01,   1.55000000e-01,
         9.00000000e-02,   5.00000000e-02,   2.50000000e-02,
         1.50000000e-02,   1.00000000e-02,   1.00000000e-02,
         1.00000000e-02,   1.00000000e-02,   1.00000000e-02,
         1.00000000e-02,   1.00000000e-02,   5.00000000e-03,
         5.00000000e-03,   5.00000000e-03,   5.00000000e-03,
         5.00000000e-03,   5.00000000e-03,   5.00000000e-03,
         5.00000000e-03,   5.00000000e-03,   5.00000000e-03,
         5.00000000e-03,   5.00000000e-03,   5.00000000e-03,
         5.00000000e-03,   5.00000000e-03])
```
I currently then have f(x) = y, and I want d^2 y / dx^2. Numerically, I know I can either interpolate the function and take the derivative analytically or use higher order finite-differences. I think that there is enough data to use either, if one or the other is considered faster, more accurate, etc. I have looked at np.interp() and scipy.interpolate with no success, as this returns me a fitted (linear or cubic) spline, but don't know how to get the derivative at that point. Any guidance is much appreciated.

Response:
You can interpolate your data using scipy's 1-D Splines functions. The computed spline has a convenient derivative method for computing derivatives. For the data of your example, using UnivariateSpline gives the following fit
```
import matplotlib.pyplot as plt
from scipy.interpolate import UnivariateSpline

y_spl = UnivariateSpline(x,y,s=0,k=4)

plt.semilogy(x,y,'ro',label = 'data')
x_range = np.linspace(x[0],x[-1],1000)
plt.semilogy(x_range,y_spl(x_range))
```
The fit seems reasonably good, at least visually. You might want to experiment with the parameters used by UnivariateSpline. The second derivate of the spline fit can be simply obtained as
```
y_spl_2d = y_spl.derivative(n=2)

plt.plot(x_range,y_spl_2d(x_range))
```
The outcome appears somewhat unnatural (in case your data corresponds to some physical process). You may either want to change the spline fit parameters, improve your data (e.g., provide more samples, perform less noisy measurements), or decide on an analytic function to model your data and perform a curve fit (e.g., using sicpy's curve_fit)


Question: Using adaptive step sizes with scipy.integrate.ode The (brief) documentation for scipy.integrate.ode says that two methods (dopri5 and dop853) have stepsize control and dense output. Looking at the examples and the code itself, I can only see a very simple way to get output from an integrator. Namely, it looks like you just step the integrator forward by some fixed dt, get the function value(s) at that time, and repeat. My problem has pretty variable timescales, so I'd like to just get the values at whatever time steps it needs to evaluate to achieve the required tolerances. That is, early on, things are changing slowly, so the output time steps can be big. But as things get interesting, the output time steps have to be smaller. I don't actually want dense output at equal intervals, I just want the time steps the adaptive function uses. EDIT: Dense output A related notion (almost the opposite) is "dense output", whereby the steps taken are as large as the stepper cares to take, but the values of the function are interpolated (usually with accuracy comparable to the accuracy of the stepper) to whatever you want. The fortran underlying scipy.integrate.ode is apparently capable of this, but ode does not have the interface. odeint, on the other hand, is based on a different code, and does evidently do dense output. (You can output every time your right-hand-side is called to see when that happens, and see that it has nothing to do with the output times.) So I could still take advantage of adaptivity, as long as I could decide on the output time steps I want ahead of time. Unfortunately, for my favorite system, I don't even know what the approximate timescales are as functions of time, until I run the integration. So I'll have to combine the idea of taking one integrator step with this notion of dense output. EDIT 2: Dense output again Apparently, scipy 1.0.0 introduced support for dense output through a new interface. In particular, they recommend moving away from scipy.integrate.odeint and towards scipy.integrate.solve_ivp, which as a keyword dense_output. If set to True, the returned object has an attribute sol that you can call with an array of times, which then returns the integrated functions values at those times. That still doesn't solve the problem for this question, but it is useful in many cases.

Response:
Since SciPy 0.13.0, The intermediate results from the dopri family of ODE solvers can now be accessed by a solout callback function.
```
import numpy as np
from scipy.integrate import ode
import matplotlib.pyplot as plt

def logistic(t, y, r):
    return r * y * (1.0 - y)

r = .01
t0 = 0
y0 = 1e-5
t1 = 5000.0

backend = 'dopri5'
# backend = 'dop853'
solver = ode(logistic).set_integrator(backend)

sol = []
def solout(t, y):
    sol.append([t, *y])
solver.set_solout(solout)
solver.set_initial_value(y0, t0).set_f_params(r)
solver.integrate(t1)

sol = np.array(sol)

plt.plot(sol[:,0], sol[:,1], 'b.-')
plt.show()
```
Result: The result seems to be slightly different from Tim D's, although they both use the same backend. I suspect this having to do with FSAL property of dopri5. In Tim's approach, I think the result k7 from the seventh stage is discarded, so k1 is calculated afresh. Note: There's a known bug with set_solout not working if you set it after setting initial values. It was fixed as of SciPy 0.17.0.


Question:
sine calculation orders of magnitude slower than cosine tl;dr Of the same numpy array, calculating np.cos takes 3.2 seconds, wheras np.sin runs 548 seconds (nine minutes) on Linux Mint. See this repo for full code. I've got a pulse signal (see image below) which I need to modulate onto a HF-carrier, simulating a Laser Doppler Vibrometer. Therefore signal and its time basis need to be resampled to match the carrier's higher sampling rate. In the following demodulation process both the in-phase carrier cos(omega * t) and the phase-shifted carrier sin(omega * t) are needed. Oddly, the time to evaluate these functions depends highly on the way the time vector has been calculated. The time vector t1 is being calculated using np.linspace directly, t2 uses the method implemented in scipy.signal.resample.
```
pulse = np.load('data/pulse.npy')  # 768 samples

pulse_samples = len(pulse)
pulse_samplerate = 960  # 960 Hz
pulse_duration = pulse_samples / pulse_samplerate  # here: 0.8 s
pulse_time = np.linspace(0, pulse_duration, pulse_samples,
                         endpoint=False)

carrier_freq = 40e6  # 40 MHz
carrier_samplerate = 100e6  # 100 MHz
carrier_samples = pulse_duration * carrier_samplerate  # 80 million

t1 = np.linspace(0, pulse_duration, carrier_samples)

# method used in scipy.signal.resample
# https://github.com/scipy/scipy/blob/v0.17.0/scipy/signal/signaltools.py#L1754
t2 = np.arange(0, carrier_samples) * (pulse_time[1] - pulse_time[0]) \
        * pulse_samples / float(carrier_samples) + pulse_time[0]
```
As can be seen in the picture below, the time vectors are not identical. At 80 million samples the difference t1 - t2 reaches 1e-8. Calculating the in-phase and shifted carrier of t1 takes 3.2 seconds each on my machine. With t2, however, calculating the shifted carrier takes 540 seconds. Nine minutes. For nearly the same 80 million values.
```
omega_t1 = 2 * np.pi * carrier_frequency * t1
np.cos(omega_t1)  # 3.2 seconds
np.sin(omega_t1)  # 3.3 seconds

omega_t2 = 2 * np.pi * carrier_frequency * t2
np.cos(omega_t2)  # 3.2 seconds
np.sin(omega_t2)  # 9 minutes
```
I can reproduce this bug on both my 32-bit laptop and my 64-bit tower, both running Linux Mint 17. On my flat mate's MacBook, however, the "slow sine" takes as little time as the other three calculations. I run a Linux Mint 17.03 on a 64-bit AMD processor and Linux Mint 17.2 on 32-bit Intel processor.

Response:
I don't think numpy has anything to do with this: I think you're tripping across a performance bug in the C math library on your system, one which affects sin near large multiples of pi. (I'm using "bug" in a pretty broad sense here -- for all I know, since the sine of large floats is poorly defined, the "bug" is actually the library behaving correctly to handle corner cases!) On linux, I get:
```
>>> %timeit -n 10000 math.sin(6e7*math.pi)
10000 loops, best of 3: 191 µs per loop
>>> %timeit -n 10000 math.sin(6e7*math.pi+0.12)
10000 loops, best of 3: 428 ns per loop
```
and other Linux-using types from the Python chatroom report
```
10000 loops, best of 3: 49.4 µs per loop
10000 loops, best of 3: 206 ns per loop
```
and
```

```python
%timeit -n 10000 math.sin(6e7*math.pi)
10000 loops, best of 3: 116 µs per loop
```

```python
%timeit -n 10000 math.sin(6e7*math.pi+0.12)
10000 loops, best of 3: 428 ns per loop
```
but a Mac user reported
```
```

```python
timeit -n 10000 math.sin(6e7*math.pi)
10000 loops, best of 3: 300 ns per loop
```

```python
%timeit -n 10000 math.sin(6e7*math.pi+0.12)
10000 loops, best of 3: 361 ns per loop
```
for no order-of-magnitude difference. As a workaround, you might try taking things mod 2 pi first:
```
>>> new = np.sin(omega_t2[-1000:] % (2*np.pi))
>>> old = np.sin(omega_t2[-1000:])
>>> abs(new - old).max()
7.83773902468434e-09
```
which has better performance:
```
>>> %timeit -n 1000 new = np.sin(omega_t2[-1000:] % (2*np.pi))
1000 loops, best of 3: 63.8 µs per loop
>>> %timeit -n 1000 old = np.sin(omega_t2[-1000:])
1000 loops, best of 3: 6.82 ms per loop
```
Note that as expected, a similar effect happens for cos, just shifted:
```
>>> %timeit -n 1000 np.cos(6e7*np.pi + np.pi/2)
1000 loops, best of 3: 37.6 µs per loop
>>> %timeit -n 1000 np.cos(6e7*np.pi + np.pi/2 + 0.12)
1000 loops, best of 3: 2.46 µs per loop
```
```


Question: skew normal distribution in scipy Does anyone know how to plot a skew normal distribution with scipy? I supose that stats.norm class can be used but I just can't figure out how. Furthermore, how can I estimate the parameters describing the skew normal distribution of a unidimensional dataset?

Response:
From the Wikipedia description,
```
from scipy import linspace
from scipy import pi,sqrt,exp
from scipy.special import erf

from pylab import plot,show

def pdf(x):
    return 1/sqrt(2*pi) * exp(-x**2/2)

def cdf(x):
    return (1 + erf(x/sqrt(2))) / 2

def skew(x,e=0,w=1,a=0):
    t = (x-e) / w
    return 2 / w * pdf(t) * cdf(a*t)
    # You can of course use the scipy.stats.norm versions
    # return 2 * norm.pdf(t) * norm.cdf(a*t)

n = 2**10

e = 1.0 # location
w = 2.0 # scale

x = linspace(-10,10,n)

for a in range(-3,4):
    p = skew(x,e,w,a)
    plot(x,p)

show()
```
If you want to find the scale, location, and shape parameters from a dataset use scipy.optimize.leastsq, for example using e=1.0,w=2.0 and a=1.0,
```
fzz = skew(x,e,w,a) + norm.rvs(0,0.04,size=n) # fuzzy data

def optm(l,x):
    return skew(x,l[0],l[1],l[2]) - fzz

print leastsq(optm,[0.5,0.5,0.5],(x,))
```
should give you something like,
```
(array([ 1.05206154,  1.96929465,  0.94590444]), 1)
```


Question:
Difference between cosine similarity and cosine distance It looks like scipy.spatial.distance.cdist cosine similariy distance: link to cos distance 1
```
1 - u*v/(||u||||v||)
```
is different from sklearn.metrics.pairwise.cosine_similarity which is link to cos similarity 2
```
u*v/||u||||v||
```
Does anybody know reason for different definitions?

Response: Good question but yes, these are 2 different things but connected by the following equation: Cosine_distance = 1 - cosine_similarity Why? Usually, people use the cosine similarity as a similarity metric between vectors. Now, the distance can be defined as 1-cos_similarity. The intuition behind this is that if 2 vectors are perfectly the same then similarity is 1 (angle=0) and thus, distance is 0 (1-1=0). Similarly you can define the cosine distance for the resulting similarity value range. Cosine similarity range: −1 meaning exactly opposite, 1 meaning exactly the same, 0 indicating orthogonality. References: Scipy wolfram


Question:
What is dimension order of numpy shape for image data? I am using nibabel lib to load data from nii file. I read the document of the lib at http://nipy.org/nibabel/gettingstarted.html, and found that This information is available without the need to load anything of the main image data into the memory. Of course there is also access to the image data as a NumPy array This is my code to load the data and it shapes
```
import nibabel as nib
img = nib.load('example.nii')
data = img.get_data()
data = np.squeeze(data)
data = np.copy(data, order="C")
print data.shape
```
I got the result
```
128, 128, 64
```
What is order of data shape? Is it WidthxHeightxDepth? And my input must arranged as depth, height, width. So I will use input=data.transpose(2,0,1). Is it right? Thanks all Update: I found that the Numpy will read the image by order Height x Width x Depth as the reference http://www.python-course.eu/images/axis.jpeg

Response:
OK, here's my take: Using scipy.ndimage.imread('img.jpg', mode='RGB'), the resulting array will always have this order: (H, W, D) i.e. (height, width, depth) because of the terminology that numpy uses for ndarrays (axis=0, axis=1, axis=2) or analogously (Y, X, Z) if one would like to visualize in 3 dimensions.
```
# read image

```python
img = scipy.ndimage.imread('suza.jpg', mode='RGB')

# image shape as (H, W, D)
```

```python
img.shape
#Output
#(634, 1366, 3)
#
## transpose to shape as (D, H, W)
```

```python
tr_img = img.transpose((-1, 0, 1))
```

```python
tr_img.shape
#Output
#(3, 634, 1366)
#```
#If you consider the img_shape as a tuple,
#```
##  index    (0,   1,    2)
#img_shape = (634, 1366, 3)
## or index  (-3,  -2,  -1)
#```
#Choose which one is a convenient way for you to remember. NOTE: The scipy.ndimage.imread() API has been removed since Scipy 1.2.0. So, it is now recommended to use imageio.imread(), which reads the image and returns Array, a subclass of numpy array, following the same conventions discussed above.
#```
## read image
#$ img = imageio.imread('suza.jpg', format='jpg')
#
## convert the image to a numpy array
#$ img_np = np.asarray(img)
#```
#PS: It should also be noted that libraries like tensorflow also (almost) follows the same convention as numpy. tf.image_decode_jpeg() returns: A Tensor of type uint8. 3-D with shape [height, width, channels]
```


Question: How do I put a constraint on SciPy curve fit? I'm trying to fit the distribution of some experimental values with a custom probability density function. Obviously, the integral of the resulting function should always be equal to 1, but the results of simple scipy.optimize.curve_fit(function, dataBincenters, dataCounts) never satisfy this condition. What is the best way to solve this problem?

Response:
You can define your own residuals function, including a penalization parameter, like detailed in the code below, where it is known beforehand that the integral along the interval must be 2.. If you test without the penalization you will see that what your are getting is the conventional curve_fit:
```
import matplotlib.pyplot as plt
import scipy
from scipy.optimize import curve_fit, minimize, leastsq
from scipy.integrate import quad
from scipy import pi, sin

x = scipy.linspace(0, pi, 100)
y = scipy.sin(x) + (0. + scipy.rand(len(x))*0.4)
def func1(x, a0, a1, a2, a3):
    return a0 + a1*x + a2*x**2 + a3*x**3

# here you include the penalization factor
def residuals(p, x, y):
    integral = quad(func1, 0, pi, args=(p[0], p[1], p[2], p[3]))[0]
    penalization = abs(2.-integral)*10000
    return y - func1(x, p[0], p[1], p[2], p[3]) - penalization

popt1, pcov1 = curve_fit(func1, x, y)
popt2, pcov2 = leastsq(func=residuals, x0=(1., 1., 1., 1.), args=(x, y))
y_fit1 = func1(x, *popt1)
y_fit2 = func1(x, *popt2)
plt.scatter(x, y, marker='.')
plt.plot(x, y_fit1, color='g', label='curve_fit')
plt.plot(x, y_fit2, color='y', label='constrained')
plt.legend()
plt.xlim(-0.1, 3.5)
plt.ylim(0, 1.4)
print('Exact integral:', quad(sin, 0, pi)[0])
print('Approx integral1:', quad(func1, 0, pi, args=(popt1[0], popt1[1], popt1[2], popt1[3]))[0])
print('Approx integral2:', quad(func1, 0, pi, args=(popt2[0], popt2[1], popt2[2], popt2[3]))[0])
plt.show()

#Exact   integral: 2.0
#Approx integral1: 2.60068579748
#Approx integral2: 2.00001911981
```
Other related questions: SciPy LeastSq Goodness of Fit Estimator


Question:
AttributeError: 'module' object (scipy) has no attribute *** Why does this error occur? In scipy, the error occurs quite often.
```

```python
import scipy
```

```python
scipy.integrate.trapz(gyroSeries, timeSeries)
#Output
#Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
#AttributeError: 'module' object has no attribute 'integrate'
#>>>
#```
#I figure out how to solve this problem by doing the following:
#```
```

```python

```

```python
import scipy.integrate
```

```python
scipy.integrate.trapz(gyroSeries, timeSeries)
```

```python
1.2
#Output
#```
#My question: Why does the error occur? Why would that fix the error?
```

Response: Most possibly because scipy is a library (package) that contains modules and to import a specific module from the scipy library, you need to specify it and import the module itself. As it's a separate module (sub-package), once you import it, it's attributes are available to you by using the regular scipy.module.attribute


Question:
Find the area between two curves plotted in matplotlib (fill_between area) I have a list of x and y values for two curves, both having weird shapes, and I don't have a function for any of them. I need to do two things: Plot it and shade the area between the curves like the image below. Find the total area of this shaded region between the curves. I'm able to plot and shade the area between those curves with fill_between and fill_betweenx in matplotlib, but I have no idea on how to calculate the exact area between them, specially because I don't have a function for any of those curves. Any ideas? I looked everywhere and can't find a simple solution for this. I'm quite desperate, so any help is much appreciated. Thank you very much! EDIT: For future reference (in case anyone runs into the same problem), here is how I've solved this: connected the first and last node/point of each curve together, resulting in a big weird-shaped polygon, then used shapely to calculate the polygon's area automatically, which is the exact area between the curves, no matter which way they go or how nonlinear they are. Works like a charm! :) Here is my code:
```py
from shapely.geometry import Polygon

x_y_curve1 = [(0.121,0.232),(2.898,4.554),(7.865,9.987)] #these are your points for curve 1 (I just put some random numbers)
x_y_curve2 = [(1.221,1.232),(3.898,5.554),(8.865,7.987)] #these are your points for curve 2 (I just put some random numbers)

polygon_points = [] #creates a empty list where we will append the points to create the polygon

for xyvalue in x_y_curve1:
    polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 1

for xyvalue in x_y_curve2[::-1]:
    polygon_points.append([xyvalue[0],xyvalue[1]]) #append all xy points for curve 2 in the reverse order (from last point to first point)

for xyvalue in x_y_curve1[0:1]:
    polygon_points.append([xyvalue[0],xyvalue[1]]) #append the first point in curve 1 again, to it "closes" the polygon

polygon = Polygon(polygon_points)
area = polygon.area
print(area)
```
EDIT 2: Thank you for the answers. Like Kyle explained, this only works for positive values. If your curves go below 0 (which is not my case, as showed in the example chart), then you would have to work with absolute numbers.

Response:
The area calculation is straightforward in blocks where the two curves don't intersect: thats the trapezium as has been pointed out above. If they intersect, then you create two triangles between x[i] and x[i+1], and you should add the area of the two. If you want to do it directly, you should handle the two cases separately. Here's a basic working example to solve your problem. First, I will start with some fake data:
```
#!/usr/bin/python
import numpy as np

# let us generate fake test data
x = np.arange(10)
y1 = np.random.rand(10) * 20
y2 = np.random.rand(10) * 20
```
Now, the main code. Based on your plot, looks like you have y1 and y2 defined at the same X points. Then we define,
```
z = y1-y2
dx = x[1:] - x[:-1]
cross_test = np.sign(z[:-1] * z[1:])
```
cross_test will be negative whenever the two graphs cross. At these points, we want to calculate the x coordinate of the crossover. For simplicity, I will calculate x coordinates of the intersection of all segments of y. For places where the two curves don't intersect, they will be useless values, and we won't use them anywhere. This just keeps the code easier to understand. Suppose you have z1 and z2 at x1 and x2, then we are solving for x0 such that z = 0:
```
# (z2 - z1)/(x2 - x1) = (z0 - z1) / (x0 - x1) = -z1/(x0 - x1)
# x0 = x1 - (x2 - x1) / (z2 - z1) * z1
x_intersect = x[:-1] - dx / (z[1:] - z[:-1]) * z[:-1]
dx_intersect = - dx / (z[1:] - z[:-1]) * z[:-1]
```
Where the curves don't intersect, area is simply given by:
```
areas_pos = abs(z[:-1] + z[1:]) * 0.5 * dx # signs of both z are same
```
Where they intersect, we add areas of both triangles:
```
areas_neg = 0.5 * dx_intersect * abs(z[:-1]) + 0.5 * (dx - dx_intersect) * abs(z[1:])
```
Now, the area in each block x[i] to x[i+1] is to be selected, for which I use np.where:
```
areas = np.where(cross_test < 0, areas_neg, areas_pos)
total_area = np.sum(areas)
```
That is your desired answer. As has been pointed out above, this will get more complicated if the both the y graphs were defined at different x points. If you want to test this, you can simply plot it (in my test case, y range will be -20 to 20)
```
negatives = np.where(cross_test < 0)
positives = np.where(cross_test >= 0)
plot(x, y1)
plot(x, y2)
plot(x, z)
plt.vlines(x_intersect[negatives], -20, 20)
```


Question: What statistics module for python supports one way ANOVA with post hoc tests (Tukey, Scheffe or other)? I have tried looking through multiple statistics modules for Python but can't seem to find any that support one-way ANOVA post hoc tests.

Response:
one way ANOVA can be used like
```
from scipy import stats
f_value, p_value = stats.f_oneway(data1, data2, data3, data4, ...)
```
This is one way ANOVA and it returns F value and P value. There is significant difference If the P value is below your setting. The Tukey-kramer HSD test can be used like
```
from statsmodels.stats.multicomp import pairwise_tukeyhsd
print pairwise_tukeyhsd(Data, Group)
```
This is multicomparison. The output is like
```
Multiple Comparison of Means - Tukey HSD,FWER=0.05
================================================
group1 group2 meandiff   lower    upper   reject
------------------------------------------------
  0      1    -35.2153 -114.8741 44.4434  False
  0      2     46.697   -40.4993 133.8932 False
  0      3    -7.5709    -87.49  72.3482  False
  1      2    81.9123    5.0289  158.7956  True
  1      3    27.6444   -40.8751  96.164  False
  2      3    -54.2679 -131.4209 22.8852  False
------------------------------------------------
```
Please refer to this site how to set the arguments. The tukeyhsd of statsmodels doesn't return P value. So, if you want to know P value, calculate from these outputted value or use R.


Question:
Numpy transpose multiplication problem I tried to find the eigenvalues of a matrix multiplied by its transpose but I couldn't do it using numpy.
```
testmatrix = numpy.array([[1,2],[3,4],[5,6],[7,8]])
prod = testmatrix * testmatrix.T
print eig(prod)
```
I expected to get the following result for the product:
```
5    11    17    23
11    25    39    53
17    39    61    83
23    53    83   113
```
and eigenvalues:
```
0.0000
0.0000
0.3929
203.6071
```
Instead I got ValueError: shape mismatch: objects cannot be broadcast to a single shape when multiplying testmatrix with its transpose. This works (the multiplication, not the code) in MatLab but I need to use it in a python application. Can someone tell me what I'm doing wrong?

Response: You might find this tutorial useful since you know MATLAB. Also, try multiplying testmatrix with the dot() function, i.e. numpy.dot(testmatrix,testmatrix.T) Apparently numpy.dot is used between arrays for matrix multiplication! The * operator is for element-wise multiplication (.* in MATLAB).


Question: Constrained Linear Regression in Python I have a classic linear regression problem of the form: y = X b where y is a response vector X is a matrix of input variables and b is the vector of fit parameters I am searching for. Python provides b = numpy.linalg.lstsq( X , y ) for solving problems of this form. However, when I use this I tend to get either extremely large or extremely small values for the components of b. I'd like to perform the same fit, but constrain the values of b between 0 and 255. It looks like scipy.optimize.fmin_slsqp() is an option, but I found it extremely slow for the size of problem I'm interested in (X is something like 3375 by 1500 and hopefully even larger). Are there any other Python options for performing constrained least squares fits? Or are there python routines for performing Lasso Regression or Ridge Regression or some other regression method which penalizes large b coefficient values?

Response: You mention you would find Lasso Regression or Ridge Regression acceptable. These and many other constrained linear models are available in the scikit-learn package. Check out the section on generalized linear models. Usually constraining the coefficients involves some kind of regularization parameter (C or alpha)---some of the models (the ones ending in CV) can use cross validation to automatically set these parameters. You can also further constrain models to use only positive coefficents---for example, there is an option for this on the Lasso model.


Question:
how do I get the subtrees of dendrogram made by scipy.cluster.hierarchy I had a confusion regarding this module (scipy.cluster.hierarchy) ... and still have some ! For example we have the following dendrogram: My question is how can I extract the coloured subtrees (each one represent a cluster) in a nice format, say SIF format ? Now the code to get the plot above is:
```
import scipy
import scipy.cluster.hierarchy as sch
import matplotlib.pylab as plt

scipy.randn(100,2)

d = sch.distance.pdist(X)

Z= sch.linkage(d,method='complete')

P =sch.dendrogram(Z)

plt.savefig('plot_dendrogram.png')

T = sch.fcluster(Z, 0.5*d.max(), 'distance')
#array([4, 5, 3, 2, 2, 3, 5, 2, 2, 5, 2, 2, 2, 3, 2, 3, 2, 5, 4, 5, 2, 5, 2,
#       3, 3, 3, 1, 3, 4, 2, 2, 4, 2, 4, 3, 3, 2, 5, 5, 5, 3, 2, 2, 2, 5, 4,
#       2, 4, 2, 2, 5, 5, 1, 2, 3, 2, 2, 5, 4, 2, 5, 4, 3, 5, 4, 4, 2, 2, 2,
#       4, 2, 5, 2, 2, 3, 3, 2, 4, 5, 3, 4, 4, 2, 1, 5, 4, 2, 2, 5, 5, 2, 2,
#       5, 5, 5, 4, 3, 3, 2, 4], dtype=int32)

sch.leaders(Z,T)
# (array([190, 191, 182, 193, 194], dtype=int32),
#  array([2, 3, 1, 4,5],dtype=int32))
```
So now, the output of fcluster() gives the clustering of the nodes (by their id's), and leaders() described here is supposed to return 2 arrays: first one contains the leader nodes of the clusters generated by Z, here we can see we have 5 clusters, as well as in the plot and the second one the id's of these clusters So if this leaders() returns resp. L and M : L[2]=182 and M[2]=1, then cluster 1 is leaded by node id 182, which doesn't exist in the observations set X, the documentation says "... then it corresponds to a non-singleton cluster". But I can't get it ... Also, I converted the Z to a tree by sch.to_tree(Z), that will return an easy-to-use tree object, which I want to visualize, but which tool should I use as a graphical platform that manipulate these kind of tree objects as inputs?

Response:
Answering the part of your question regarding tree manipulation... As explained in another answer, you can read the coordinates of the branches reading icoord and dcoord from the tree object. For each branch the coordinated are given from the left to the right. If you want to manually plot the tree you can use something like:
```
def plot_tree(P, pos=None):
    plt.clf()
    icoord = scipy.array(P['icoord'])
    dcoord = scipy.array(P['dcoord'])
    color_list = scipy.array(P['color_list'])
    xmin, xmax = icoord.min(), icoord.max()
    ymin, ymax = dcoord.min(), dcoord.max()
    if pos:
        icoord = icoord[pos]
        dcoord = dcoord[pos]
        color_list = color_list[pos]
    for xs, ys, color in zip(icoord, dcoord, color_list):
        plt.plot(xs, ys, color)
    plt.xlim(xmin-10, xmax + 0.1*abs(xmax))
    plt.ylim(ymin, ymax + 0.1*abs(ymax))
    plt.show()
```
Where, in your code, plot_tree(P) gives: The function allows you to select just some branches:
```
plot_tree(P, range(10))
```
Now you have to know which branches to plot. Maybe the fcluster() output is a little obscure and another way to find which branches to plot based on a minimum and a maximum distance tolerance would be using the output of linkage() directly (Z in the OP's case):
```
dmin = 0.2
dmax = 0.3
pos = scipy.all( (Z[:,2] >= dmin, Z[:,2] <= dmax), axis=0 ).nonzero()
plot_tree( P, pos )
```
Recommended references: How does condensed distance matrix work? (pdist) how to plot and annotate hierarchical clustering dendrograms in scipy/matplotlib


Question: savetxt How change the type from float64 to int or double I have been trying to use the savetxt function in numpy. The problem I am running into is that even thought I define my variables accordingly, i.e. int() or double(), the text file i am getting out has floats in them. How can I change that? Input is as follows: pNoise=[int(i), around(pNoise[0], decimals=3), around(pNoise[1], decimals=3), around(pNoise[2], decimals=3)] savetxt line is as follows: savetxt(noutF, pNoisetot) What I expect is: 0 1.567 8.865 instead I get 0.000000000000000000e+00 1.015909999999999940e+02 2.600000000000000089e-01

Response:
You can define how the output has to be formatted with the fmt parameter of np.savetxt, e.g.: for floats rounded to five decimals:
```
np.savetxt("file.txt", output, fmt='%10.5f', delimiter='\t')
```
for integers:
```
np.savetxt("file.txt", output, fmt='%i', delimiter='\t')
```
Here you can find more information about the possibilities of fmt: http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html


Question: How to disregard the NaN data point in numpy array and generate the normalized data in Python? Say I have a numpy array that has some float('nan'), I don't want to impute those data now and I want to first normalize those and keep the NaN data at the original space, is there any way I can do that? Previously I used normalize function in sklearn.Preprocessing, but that function seems can't take any NaN contained array as input.

Response:
You can mask your array using the numpy.ma.array function and subsequently apply any numpy operation:
```
import numpy as np

a = np.random.rand(10)            # Generate random data.
a = np.where(a > 0.8, np.nan, a)  # Set all data larger than 0.8 to NaN

a = np.ma.array(a, mask=np.isnan(a)) # Use a mask to mark the NaNs

a_norm  = a / np.sum(a) # The sum function ignores the masked values.
a_norm2 = a / np.std(a) # The std function ignores the masked values.
```
You can still access your raw data:
```
print a.data
```


Question: Scientific libraries for Lua? [closed] Closed. This question is seeking recommendations for software libraries, tutorials, tools, books, or other off-site resources. It does not meet Stack Overflow guidelines. It is not currently accepting answers. We don’t allow questions seeking recommendations for software libraries, tutorials, tools, books, or other off-site resources. You can edit the question so it can be answered with facts and citations. Closed 10 years ago. Improve this question Are there any scientific packages for Lua comparable to Scipy?

Response: You should try Torch7 (github). Torch7 has a very nice and efficient vector/matrix/tensor numerical library with a Lua front-end. It also has a bunch of functions for computer vision and machine learning. It's pretty recent but getting better quickly.


Question:
Why isn't `curve_fit` able to estimate the covariance of the parameter if the parameter fits exactly? I don't understand curve_fit isn't able to estimate the covariance of the parameter, thus raising the OptimizeWarning below. The following MCVE explains my problem: MCVE python snippet
```
from scipy.optimize import curve_fit
func = lambda x, a: a * x
popt, pcov = curve_fit(f = func, xdata = [1], ydata = [1])
print(popt, pcov)
```
Output
```
\python-3.4.4\lib\site-packages\scipy\optimize\minpack.py:715:
OptimizeWarning: Covariance of the parameters could not be estimated
category=OptimizeWarning)

[ 1.] [[ inf]]
```
For a = 1 the function fits xdata and ydata exactly. Why isn't the error/variance 0, or something close to 0, but inf instead? There is this quote from the curve_fit SciPy Reference Guide: If the Jacobian matrix at the solution doesn’t have a full rank, then ‘lm’ method returns a matrix filled with np.inf, on the other hand ‘trf’ and ‘dogbox’ methods use Moore-Penrose pseudoinverse to compute the covariance matrix. So, what's the underlying problem? Why doesn't the Jacobian matrix at the solution have a full rank?

Response: The formula for the covariance of the parameters (Wikipedia) has the number of degrees of freedom in the denominator. The degrees of freedoms are computed as (number of data points) - (number of parameters), which is 1 - 1 = 0 in your example. And this is where SciPy checks the number of degrees of freedom before dividing by it. With xdata = [1, 2], ydata = [1, 2] you would get zero covariance (note that the model still fits exactly: exact fit is not the problem). This is the same sort of issue as sample variance being undefined if the sample size N is 1 (the formula for sample variance has (N-1) in the denominator). If we only took size=1 sample out of the population, we don't estimate the variance by zero, we know nothing about the variance.


Question:
TypeError: cannot unpack non-iterable int objec How can I solve this error After running my code as follows . I am using the function below and implementin running window for loop on it but end up getting the error below. The for loop works and hungs at a point.
```
def get_grps(s, thresh=-1, Nmin=3):
    """
    Nmin : int > 0
    Min number of consecutive values below threshold.
    """
    m = np.logical_and.reduce([s.shift(-i).le(thresh) for i in range(Nmin)])
    if Nmin > 1:
        m = pd.Series(m, index=s.index).replace({False: np.NaN}).ffill(limit=Nmin - 1).fillna(False)
    else:
        m = pd.Series(m, index=s.index)

    # Form consecutive groups
    gps = m.ne(m.shift(1)).cumsum().where(m)

    # Return None if no groups, else the aggregations
    if gps.isnull().all():
        return 0
    else:
        agg = s.groupby(gps).agg([list, sum, 'size']).reset_index(drop=True)
        # agg2 = s2.groupby(gps).agg([list, sum, 'size']).reset_index(drop=True)
        return agg, gps

data_spi = [-0.32361498 -0.5229471   0.15702732  0.28753752   -0.01069884 -0.8163699
  -1.3169327   0.4413181   0.75815576  1.3858147   0.49990863-0.06357133
-0.78432    -0.95337325 -1.663739    0.18965477  0.81183237   0.8360347
  0.99537593 -0.12197364 -0.31432647 -2.0865853   0.2084263    0.13332903
 -0.05270813 -1.0090573  -1.6578217  -1.2969246  -0.70916456   0.70059913
 -1.2127264  -0.659762   -1.1612778  -2.1216285  -0.8054617    -0.6293912
 -2.2103117  -1.9373081  -2.530625   -2.4089663  -1.950846    -1.6129876]
lon = data_spi.lon
lat = data_spi.lat
print(len(data_spi))

n=6

for x in range(len(lat)):
    for y in range(len(lon)):
        if data_spi[0, x, y] != 0:
            for i in range(len(data_spi)-70):
                ts = data_spi[i:i+10, x, y].fillna(1)
                print(ts)
                # print(np.array(ts))

                agg, gps = get_grps(pd.Series(ts), thresh=-1, Nmin=3)

                duration = np.nanmean(agg['sum'])
                frequency = len(agg['sum'])
                severity = np.abs(np.mean(agg['sum']))
                intensity = np.mean(np.abs(agg['sum'] / agg['size']))
                print(f'intensity {intensity}')
```
I get this error
```
Traceback (most recent call last):
 File "/Users/mada0007/PycharmProjects/Research_ass /FREQ_MEAN_INT_DUR_CORR.py", line 80, in <module>
 agg, gps = get_grps(pd.Series(ts), thresh=-1, Nmin=3)
 typeError: cannot unpack non-iterable int object
```
How can I resolve this error?

Response:
Just replace return 0 by return 0, 0, or better: raise an error instead of returning 0 When your if condition is True, you only return 0. Then later, when you do agg, gps = get_grps(...), you tell python to unpack the result of the function. Then, python is expecting a 2-length iterable, and try to unpack it, but as it says: it 'cannot unpack non-iterable int object'... So a quick workaround is to return a tuple (0, 0) with return 0, 0, but it is quite bad because you return integers where objects are expected. your script will crash on the next line duration = np.nanmean(agg['sum']) (since agg is 0). Some cleaner solutions to handle this case would be to unpack in a second time:
```
def get_grps(s, thresh=-1, Nmin=3):
    # ...
    if gps.isnull().all():
        return None
    else:
        # ...
        return agg, gps

for i in range(len(data_spi)-70):
    ts = data_spi[i:i+10, x, y].fillna(1)

    result = get_grps(pd.Series(ts), thresh=-1, Nmin=3)
    if result is None:
        break

    agg, gps = result

    duration = np.nanmean(agg['sum'])
    frequency = len(agg['sum'])
```


Question: High Pass Filter for image processing in python by using scipy/numpy I am currently studying image processing. In SciPy, I know there is a median filter in scipy.signal. Is there a filter similar to a high pass filter?

Response:
"High pass filter" is a very generic term. There are an infinite number of different "highpass filters" that do very different things (e.g. an edge dectection filter, as mentioned earlier, is technically a highpass (most are actually a bandpass) filter, but has a very different effect from what you probably had in mind.) At any rate, based on most of the questions you've been asking, you should probably look into scipy.ndimage instead of scipy.filter, especially if you're going to be working with large images (ndimage can preform operations in-place, conserving memory). As a basic example, showing a few different ways of doing things:
```
import matplotlib.pyplot as plt
import numpy as np
from scipy import ndimage
import Image

def plot(data, title):
    plot.i += 1
    plt.subplot(2,2,plot.i)
    plt.imshow(data)
    plt.gray()
    plt.title(title)
plot.i = 0

# Load the data...
im = Image.open('lena.png')
data = np.array(im, dtype=float)
plot(data, 'Original')

# A very simple and very narrow highpass filter
kernel = np.array([[-1, -1, -1],
                   [-1,  8, -1],
                   [-1, -1, -1]])
highpass_3x3 = ndimage.convolve(data, kernel)
plot(highpass_3x3, 'Simple 3x3 Highpass')

# A slightly "wider", but sill very simple highpass filter
kernel = np.array([[-1, -1, -1, -1, -1],
                   [-1,  1,  2,  1, -1],
                   [-1,  2,  4,  2, -1],
                   [-1,  1,  2,  1, -1],
                   [-1, -1, -1, -1, -1]])
highpass_5x5 = ndimage.convolve(data, kernel)
plot(highpass_5x5, 'Simple 5x5 Highpass')

# Another way of making a highpass filter is to simply subtract a lowpass
# filtered image from the original. Here, we'll use a simple gaussian filter
# to "blur" (i.e. a lowpass filter) the original.
lowpass = ndimage.gaussian_filter(data, 3)
gauss_highpass = data - lowpass
plot(gauss_highpass, r'Gaussian Highpass, $\sigma = 3 pixels$')

plt.show()
```


Question:
Multidimensional Euclidean Distance in Python I want to calculate the Euclidean distance in multiple dimensions (24 dimensions) between 2 arrays. I'm using numpy-Scipy. Here is my code:
```
import numpy,scipy;

A=numpy.array([116.629, 7192.6, 4535.66, 279714, 176404, 443608, 295522, 1.18399e+07, 7.74233e+06, 2.85839e+08, 2.30168e+08, 5.6919e+08, 168989, 7.48866e+06, 1.45261e+06, 7.49496e+07, 2.13295e+07, 3.74361e+08, 54.5, 3349.39, 262.614, 16175.8, 3693.79, 205865]);

B=numpy.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 151246, 6795630, 4566625, 2.0355328e+08, 1.4250515e+08, 3.2699482e+08, 95635, 4470961, 589043, 29729866, 6124073, 222.3]);
```
However, I used scipy.spatial.distance.cdist(A[numpy.newaxis,:],B,'euclidean') to calcuate the eucleidan distance. But it gave me an error
```
raise ValueError('XB must be a 2-dimensional array.');
```
I don't seem to understand it. I looked up scipy.spatial.distance.pdist but don't understand how to use it? Is there any other better way to do it?

Response:
Perhaps scipy.spatial.distance.euclidean? Examples
```

```python
from scipy.spatial import distance
```

```python
distance.euclidean([1, 0, 0], [0, 1, 0])
#Output
#1.4142135623730951
```

```python
distance.euclidean([1, 1, 0], [0, 1, 0])
#Output
#1.0
#```
```


Question: multidimensional confidence intervals [closed] Closed. This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Guide the asker to update the question so it focuses on a single, specific problem. Narrowing the question will help others answer the question concisely. You may edit the question if you feel you can improve it yourself. If edited, the question will be reviewed and might be reopened. Closed 4 years ago. Improve this question I have numerous tuples (par1,par2), i.e. points in a 2 dimensional parameter space obtained from repeating an experiment multiple times. I'm looking for a possibility to calculate and visualize confidence ellipses (not sure if thats the correct term for this). Here an example plot that I found in the web to show what I mean: source: blogspot.ch/2011/07/classification-and-discrimination-with.html So in principle one has to fit a multivariate normal distribution to a 2D histogram of data points I guess. Can somebody help me with this?

Response:
It sounds like you just want the 2-sigma ellipse of the scatter of points? If so, consider something like this (From some code for a paper here: https://github.com/joferkington/oost_paper_code/blob/master/error_ellipse.py):
```
import numpy as np

import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse

def plot_point_cov(points, nstd=2, ax=None, **kwargs):
    """
    Plots an `nstd` sigma ellipse based on the mean and covariance of a point
    "cloud" (points, an Nx2 array).

    Parameters
    ----------
        points : An Nx2 array of the data points.
        nstd : The radius of the ellipse in numbers of standard deviations.
            Defaults to 2 standard deviations.
        ax : The axis that the ellipse will be plotted on. Defaults to the
            current axis.
        Additional keyword arguments are pass on to the ellipse patch.

    Returns
    -------
        A matplotlib ellipse artist
    """
    pos = points.mean(axis=0)
    cov = np.cov(points, rowvar=False)
    return plot_cov_ellipse(cov, pos, nstd, ax, **kwargs)

def plot_cov_ellipse(cov, pos, nstd=2, ax=None, **kwargs):
    """
    Plots an `nstd` sigma error ellipse based on the specified covariance
    matrix (`cov`). Additional keyword arguments are passed on to the
    ellipse patch artist.

    Parameters
    ----------
        cov : The 2x2 covariance matrix to base the ellipse on
        pos : The location of the center of the ellipse. Expects a 2-element
            sequence of [x0, y0].
        nstd : The radius of the ellipse in numbers of standard deviations.
            Defaults to 2 standard deviations.
        ax : The axis that the ellipse will be plotted on. Defaults to the
            current axis.
        Additional keyword arguments are pass on to the ellipse patch.

    Returns
    -------
        A matplotlib ellipse artist
    """
    def eigsorted(cov):
        vals, vecs = np.linalg.eigh(cov)
        order = vals.argsort()[::-1]
        return vals[order], vecs[:,order]

    if ax is None:
        ax = plt.gca()

    vals, vecs = eigsorted(cov)
    theta = np.degrees(np.arctan2(*vecs[:,0][::-1]))

    # Width and height are "full" widths, not radius
    width, height = 2 * nstd * np.sqrt(vals)
    ellip = Ellipse(xy=pos, width=width, height=height, angle=theta, **kwargs)

    ax.add_artist(ellip)
    return ellip

if __name__ == '__main__':
    #-- Example usage -----------------------
    # Generate some random, correlated data
    points = np.random.multivariate_normal(
            mean=(1,1), cov=[[0.4, 9],[9, 10]], size=1000
            )
    # Plot the raw points...
    x, y = points.T
    plt.plot(x, y, 'ro')

    # Plot a transparent 3 standard deviation covariance ellipse
    plot_point_cov(points, nstd=3, alpha=0.5, color='green')

    plt.show()
```


Question:
How to filter/smooth with SciPy/Numpy? I am trying to filter/smooth signal obtained from a pressure transducer of sampling frequency 50 kHz. A sample signal is shown below: I would like to obtain a smooth signal obtained by loess in MATLAB (I am not plotting the same data, values are different). I calculated the power spectral density using matplotlib's psd() function and the power spectral density is also provided below: I have tried using the following code and obtained a filtered signal:
```
import csv
import numpy as np
import matplotlib.pyplot as plt
import scipy as sp
from scipy.signal import butter, lfilter, freqz

def butter_lowpass(cutoff, fs, order=5):
    nyq = 0.5 * fs
    normal_cutoff = cutoff / nyq
    b, a = butter(order, normal_cutoff, btype='low', analog=False)
    return b, a

def butter_lowpass_filter(data, cutoff, fs, order=5):
    b, a = butter_lowpass(cutoff, fs, order=order)
    y = lfilter(b, a, data)
    return y

data = np.loadtxt('data.dat', skiprows=2, delimiter=',', unpack=True).transpose()
time = data[:,0]
pressure = data[:,1]
cutoff = 2000
fs = 50000
pressure_smooth = butter_lowpass_filter(pressure, cutoff, fs)

figure_pressure_trace = plt.figure(figsize=(5.15, 5.15))
figure_pressure_trace.clf()
plot_P_vs_t = plt.subplot(111)
plot_P_vs_t.plot(time, pressure, linewidth=1.0)
plot_P_vs_t.plot(time, pressure_smooth, linewidth=1.0)
plot_P_vs_t.set_ylabel('Pressure (bar)', labelpad=6)
plot_P_vs_t.set_xlabel('Time (ms)', labelpad=6)
plt.show()
plt.close()
```
The output I get is: I need more smoothing, I tried changing the cutoff frequency but still satisfactory results can not be obtained. I can't get the same smoothness by MATLAB. I am sure it can be done in Python, but how? You can find the data here. Update I applied lowess smoothing from statsmodels, this also does not provide satisfactory results.

Response:
Here are a couple suggestions. First, try the lowess function from statsmodels with it=0, and tweak the frac argument a bit:
```

```python
from statsmodels.nonparametric.smoothers_lowess import lowess
```

```python
filtered = lowess(pressure, time, is_sorted=True, frac=0.025, it=0)
```

```python
plot(time, pressure, 'r')
#Output
#[<matplotlib.lines.Line2D at 0x1178d0668>]
```

```python
plot(filtered[:,0], filtered[:,1], 'b')
#Output
#[<matplotlib.lines.Line2D at 0x1173d4550>]
#```
#A second suggestion is to use scipy.signal.filtfilt instead of lfilter to apply the Butterworth filter. filtfilt is the forward-backward filter. It applies the filter twice, once forward and once backward, resulting in zero phase delay. Here's a modified version of your script. The significant changes are the use of filtfilt instead of lfilter, and the change of cutoff from 3000 to 1500. You might want to experiment with that parameter--higher values result in better tracking of the onset of the pressure increase, but a value that is too high doesn't filter out the 3kHz (roughly) oscillations after the pressure increase.
#```
#import numpy as np
#import matplotlib.pyplot as plt
#from scipy.signal import butter, filtfilt
#
#def butter_lowpass(cutoff, fs, order=5):
#    nyq = 0.5 * fs
#    normal_cutoff = cutoff / nyq
#    b, a = butter(order, normal_cutoff, btype='low', analog=False)
#    return b, a
#
#def butter_lowpass_filtfilt(data, cutoff, fs, order=5):
#    b, a = butter_lowpass(cutoff, fs, order=order)
#    y = filtfilt(b, a, data)
#    return y
#
#data = np.loadtxt('data.dat', skiprows=2, delimiter=',', unpack=True).transpose()
#time = data[:,0]
#pressure = data[:,1]
#cutoff = 1500
#fs = 50000
#pressure_smooth = butter_lowpass_filtfilt(pressure, cutoff, fs)
#
#figure_pressure_trace = plt.figure()
#figure_pressure_trace.clf()
#plot_P_vs_t = plt.subplot(111)
#plot_P_vs_t.plot(time, pressure, 'r', linewidth=1.0)
#plot_P_vs_t.plot(time, pressure_smooth, 'b', linewidth=1.0)
#plt.show()
#plt.close()
#```
#Here's the plot of the result. Note the deviation in the filtered signal at the right edge. To handle that, you can experiment with the padtype and padlen parameters of filtfilt, or, if you know you have enough data, you can discard the edges of the filtered signal.
```


Question:
numpy array creating with a sequence I am on my transitional trip from MATLAB to scipy(+numpy)+matplotlib. I keep having issues when implementing some things. I want to create a simple vector array in three different parts. In MATLAB I would do something like:
```
vector=[0.2,1:60,60.8];
```
This results in a one dimensional array of 62 positions. I'm trying to implement this using scipy. The closest I am right now is this:
```
a=[[0.2],linspace(1,60,60),[60.8]]
```
However this creates a list, not an array, and hence I cannot reshape it to a vector array. But then, when I do this, I get an error
```
a=array([[0.2],linspace(1,60,60),[60.8]])
ValueError: setting an array element with a sequence.
```
I believe my main obstacle is that I can't figure out how to translate this simple operation in MATLAB:
```
a=[1:2:20];
```
to numpy. I know how to do it to access positions in an array, although not when creating a sequence. Any help will be appreciated, thanks!

Response:
Well NumPy implements MATLAB's array-creation function, vector, using two functions instead of one--each implicitly specifies a particular axis along which concatenation ought to occur. These functions are: r_ (row-wise concatenation) and c_ (column-wise) So for your example, the NumPy equivalent is:
```

```python
import numpy as NP
```

```python
v = NP.r_[.2, 1:10, 60.8]
```

```python
print(v)
#Output
#     [  0.2   1.    2.    3.    4.    5.    6.    7.    8.    9.   60.8]
#```
#The column-wise counterpart is:
#```
```

```python
NP.c_[.2, 1:10, 60.8]
#Output
#```
#slice notation works as expected [start:stop:step]:
#```
```

```python
v = NP.r_[.2, 1:25:7, 60.8]
```

```python
v
#Output
#  array([  0.2,   1. ,   8. ,  15. ,  22. ,  60.8])
#```
#Though if an imaginary number of used as the third argument, the slicing notation behaves like linspace:
#```
```

```python
v = NP.r_[.2, 1:25:7j, 60.8]
```

```python
v
#Output
#  array([  0.2,   1. ,   5. ,   9. ,  13. ,  17. ,  21. ,  25. ,  60.8])
#```
#Otherwise, it behaves like arange:
#```
```

```python
v = NP.r_[.2, 1:25:7, 60.8]
```

```python
v
#Output
#  array([  0.2,   1. ,   8. ,  15. ,  22. ,  60.8])
#```
```


Question:
Pass args for solve_ivp (new SciPy ODE API) For solving simple ODEs using SciPy, I used to use the odeint function, with form:
```
scipy.integrate.odeint(func, y0, t, args=(), Dfun=None, col_deriv=0, full_output=0, ml=None, mu=None, rtol=None, atol=None, tcrit=None, h0=0.0, hmax=0.0, hmin=0.0, ixpr=0, mxstep=0, mxhnil=0, mxordn=12, mxords=5, printmessg=0)[source]
```
where a simple function to be integrated could include additional arguments of the form:
```
def dy_dt(t, y, arg1, arg2):
    # processing code here
```
In SciPy 1.0, it seems the ode and odeint funcs have been replaced by a newer solve_ivp method.
```
scipy.integrate.solve_ivp(fun, t_span, y0, method='RK45', t_eval=None, dense_output=False, events=None, vectorized=False, **options)
```
However, this doesn't seem to offer an args parameter, nor any indication in the documentation as to implementing the passing of args. Therefore, I wonder if arg passing is possible with the new API, or is this a feature that has yet to be added? (It would seem an oversight to me if this features has been intentionally removed?) Reference: https://docs.scipy.org/doc/scipy/reference/integrate.html

Response:
Relatively recently there appeared a similar question on scipy's github. Their solution is to use lambda:
```
solve_ivp(fun=lambda t, y: fun(t, y, *args), ...)
```
And they argue that there is already enough overhead for this not to matter.


Question:
Matlab vs Python: Reshape So I found this: When converting MATLAB code it might be necessary to first reshape a matrix to a linear sequence, perform some indexing operations and then reshape back. As reshape (usually) produces views onto the same storage, it should be possible to do this fairly efficiently. Note that the scan order used by reshape in Numpy defaults to the 'C' order, whereas MATLAB uses the Fortran order. If you are simply converting to a linear sequence and back this doesn't matter. But if you are converting reshapes from MATLAB code which relies on the scan order, then this MATLAB code:
```
z = reshape(x,3,4);
```
should become
```
z = x.reshape(3,4,order='F').copy()
```
in Numpy. I have a multidimensional 16*2 array called mafs, when I do in MATLAB:
```
mafs2 = reshape(mafs,[4,4,2])
```
I get something different than when in python I do:
```
mafs2 = reshape(mafs,(4,4,2))
```
or even
```
mafs2 = mafs.reshape((4,4,2),order='F').copy()
```
Any help on this? Thank you all.

Response:
Example: MATLAB:
```
>> mafs = [(1:16)' (17:32)']
mafs =
     1    17
     2    18
     3    19
     4    20
     5    21
     6    22
     7    23
     8    24
     9    25
    10    26
    11    27
    12    28
    13    29
    14    30
    15    31
    16    32

>> reshape(mafs,[4 4 2])
ans(:,:,1) =
     1     5     9    13
     2     6    10    14
     3     7    11    15
     4     8    12    16
ans(:,:,2) =
    17    21    25    29
    18    22    26    30
    19    23    27    31
    20    24    28    32
```
Python:
```

```python
import numpy as np
```

```python
mafs = np.c_[np.arange(1,17), np.arange(17,33)]
```

```python
mafs.shape
#Output
#(16, 2)
```

```python
mafs[:,0]
#Output
#array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16])
```

```python
mafs[:,1]
#Output
#array([17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32])
```

```python
r = np.reshape(mafs, (4,4,2), order="F")
```

```python
r.shape
#Output
#(4, 4, 2)
```

```python
r[:,:,0]
#Output
#array([[ 1,  5,  9, 13],
#       [ 2,  6, 10, 14],
#       [ 3,  7, 11, 15],
#       [ 4,  8, 12, 16]])
```

```python
r[:,:,1]
#Output
#array([[17, 21, 25, 29],
#       [18, 22, 26, 30],
#       [19, 23, 27, 31],
#       [20, 24, 28, 32]])
#```
```


Question:
Clustering geo location coordinates (lat,long pairs) using KMeans algorithm with Python Using the following code to cluster geolocation coordinates results in 3 clusters:
```
import numpy as np
    import matplotlib.pyplot as plt
    from scipy.cluster.vq import kmeans2, whiten

    coordinates= np.array([
               [lat, long],
               [lat, long],
                ...
               [lat, long]
               ])
    x, y = kmeans2(whiten(coordinates), 3, iter = 20)
    plt.scatter(coordinates[:,0], coordinates[:,1], c=y);
    plt.show()
```
Is it right to use Kmeans for location clustering, as it uses Euclidean distance and not Haversine formula as a distance function?

Response:
k-means is not a good algorithm to use for spatial clustering, for the reasons you meantioned. Instead, you could do this clustering job using scikit-learn's DBSCAN with the haversine metric and ball-tree algorithm. This tutorial demonstrates clustering latitude-longitude spatial data with DBSCAN/haversine and avoids all those Euclidean-distance problems:
```
df = pd.read_csv('gps.csv')
coords = df.as_matrix(columns=['lat', 'lon'])
db = DBSCAN(eps=eps, min_samples=ms, algorithm='ball_tree', metric='haversine').fit(np.radians(coords))
```
Note that this specifically uses scikit-learn v0.15, as some earlier/later versions seem to require a full distance matrix to be computed. Also notice that the eps value is in radians and that .fit() takes the coordinates in radian units for the haversine metric.


Question: How do I apply a DCT to an image in Python? I want to apply a Discrete Cosine Transform (as well as the inverse) to an image in Python and I'm wondering what is the best way to do it and how. I've looked at PIL and OpenCV but I still don't understand how to use it.

Response:
Example with scipy.fftpack:
```
from scipy.fftpack import dct, idct

# implement 2D DCT
def dct2(a):
    return dct(dct(a.T, norm='ortho').T, norm='ortho')

# implement 2D IDCT
def idct2(a):
    return idct(idct(a.T, norm='ortho').T, norm='ortho')

from skimage.io import imread
from skimage.color import rgb2gray
import numpy as np
import matplotlib.pylab as plt

# read lena RGB image and convert to grayscale
im = rgb2gray(imread('images/lena.jpg'))
imF = dct2(im)
im1 = idct2(imF)

# check if the reconstructed image is nearly equal to the original image
np.allclose(im, im1)
# True

# plot original and reconstructed images with matplotlib.pylab
plt.gray()
plt.subplot(121), plt.imshow(im), plt.axis('off'), plt.title('original image', size=20)
plt.subplot(122), plt.imshow(im1), plt.axis('off'), plt.title('reconstructed image (DCT+IDCT)', size=20)
plt.show()
```
Also, if you plot a small slice of the 2D DCT coefficients array imF (in log domain), you will get a figure like the following (with a checkerboard pattern):


Question:
Scipy.sparse.csr_matrix: How to get top ten values and indices? I have a large csr_matrix and I am interested in the top ten values and their indices each row. But I did not find a decent way to manipulate the matrix. Here is my current solution and the main idea is to process them row by row:
```
row = csr_matrix.getrow(row_number).toarray()[0].ravel()
top_ten_indicies = row.argsort()[-10:]
top_ten_values = row[row.argsort()[-10:]]
```
By doing this, the advantages of csr_matrix is not fully used. It's more like a brute force solution.

Response:
I don't see what the advantages of csr format are in this case. Sure, all the nonzero values are collected in one .data array, with the corresponding column indexes in .indices. But they are in blocks of varying length. And that means they can't be processed in parallel or with numpy array strides. One solution is the pad those blocks into common length blocks. That's what .toarray() does. Then you can find the maximum values with argsort(axis=1) or withargpartition`. Another is to break them into row sized blocks, and process each of those. That's what you are doing with the .getrow. Another way of breaking them up is convert to lil format, and process the sublists of the .data and .rows arrays. A possible third option is to use the ufunc reduceat method. This lets you apply ufunc reduction methods to sequential blocks of an array. There are established ufunc like np.add that take advantage of this. argsort is not such a function. But there is a way of constructing a ufunc from a Python function, and gain some modest speed over regular Python iteration. [I need to look up a recent SO question that illustrates this.] I'll illustrate some of this with a simpler function, sum over rows. If A2 is a csr matrix.
```
A2.sum(axis=1)  # the fastest compile csr method
A2.A.sum(axis=1)  # same, but with a dense intermediary
[np.sum(l.data) for l in A2]  # iterate over the rows of A2
[np.sum(A2.getrow(i).data) for i in range(A2.shape[0])]  # iterate with index
[np.sum(l) for l in A2.tolil().data]  # sum the sublists of lil format
np.add.reduceat(A2.data, A2.indptr[:-1])  # with reduceat
```
A2.sum(axis=1) is implemented as a matrix multiplication. That's not relevant to the sort problem, but still an interesting way of looking at the summation problem. Remember csr format was developed for efficient multiplication. For a my current sample matrix (created for another SO sparse question)
```
<8x47752 sparse matrix of type '<class 'numpy.float32'>'
     with 32 stored elements in Compressed Sparse Row format>
```
some comparative times are
```

```python
timeit np.add.reduceat(A2.data, A2.indptr[:-1])
100000 loops, best of 3: 7.41 µs per loop
```

```python
timeit A2.sum(axis=1)
10000 loops, best of 3: 71.6 µs per loop
```

```python
timeit [np.sum(l) for l in A2.tolil().data]
1000 loops, best of 3: 280 µs per loop
```
Everything else is 1ms or more. I suggest focusing on developing your one-row function, something like:
```
def max_n(row_data, row_indices, n):
    i = row_data.argsort()[-n:]
    # i = row_data.argpartition(-n)[-n:]
    top_values = row_data[i]
    top_indices = row_indices[i]  # do the sparse indices matter?
    return top_values, top_indices, i
```
Then see how if fits in one of these iteration methods. tolil() looks most promising. I haven't addressed the question of how to collect these results. Should they be lists of lists, array with 10 columns, another sparse matrix with 10 values per row, etc.? sorting each row of a large sparse & saving top K values & column index - Similar question from several years back, but unanswered. Argmax of each row or column in scipy sparse matrix - Recent question seeking argmax for rows of csr. I discuss some of the same issues. how to speed up loop in numpy? - example of how to use np.frompyfunc to create a ufunc. I don't know if the resulting function has the .reduceat method. Increasing value of top k elements in sparse matrix - get the top k elements of csr (not by row). Case for argpartition. The row summation implemented with np.frompyfunc:
```
```

```python
def foo(a,b):
    return a+b
```

```python
vfoo=np.frompyfunc(foo,2,1)
```

```python
timeit vfoo.reduceat(A2.data,A2.indptr[:-1],dtype=object).astype(float)
10000 loops, best of 3: 26.2 µs per loop
```
That's respectable speed. But I can't think of a way of writing a binary function (takes to 2 arguments) that would implement argsort via reduction. So this is probably a deadend for this problem.
```


Question: Positive directional derivative for linesearch What does the smode of scipy.optimize 'Positive directional derivative for linesearch' mean? for example in fmin_slsqp http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html

Response: These optimization algorithms typically work by choosing a descent direction, and then performing a line search to that direction. I think this message means that the optimizer got into a position where it did not manage to find a direction where the value of the objective function decreases (fast enough), but could also not verify that the current position is a minimum.


Question:
harmonic mean in python The Harmonic Mean function in Python (scipy.stats.hmean) requires that the input be positive numbers. For example:
```
from scipy import stats
print stats.hmean([ -50.2 , 100.5 ])
```
results in:
```
ValueError: Harmonic mean only defined if all elements greater than zero
```
I don't mathematically see why this should be the case, except for the rare instance where you would end up dividing by zero. Instead of checking for a divide by zero, hmean() then throws an error upon inputing any positive number, whether a harmonic mean can be found or not. Am I missing something here in the maths? Or is this really a limitation in SciPy? How would you go about finding the harmonic mean of a set of numbers which might be positive or negative in python?

Response: The harmonic mean is only defined for sets of positive real numbers. If you try and compute it for sets with negatives you get all kinds of strange and useless results even if you don't hit div by 0. For example, applying the formula to the set (3, -3, 4) gives a mean of 12!


Question:
Custom cluster colors of SciPy dendrogram in Python (link_color_func?) I want to color my clusters with a color map that I made in the form of a dictionary (i.e. {leaf: color}). I've tried following https://joernhees.de/blog/2015/08/26/scipy-hierarchical-clustering-and-dendrogram-tutorial/ but the colors get messed up for some reason. The default plot looks good, I just want to assign those colors differently. I saw that there was a link_color_func but when I tried using my color map (D_leaf_color dictionary) I got an error b/c it wasn't a function. I've created D_leaf_color to customize the colors of the leaves associated with particular clusters. In my actual dataset, the colors mean something so I'm steering away from arbitrary color assignments. I don't want to use color_threshold b/c in my actual data, I have way more clusters and SciPy repeats the colors, hence this question. . . How can I use my leaf-color dictionary to customize the color of my dendrogram clusters? I made a GitHub issue https://github.com/scipy/scipy/issues/6346 where I further elaborated on the approach to color the leaves in Interpreting the output of SciPy's hierarchical clustering dendrogram? (maybe found a bug...) but I still can't figure out how to actually either: (i) use dendrogram output to reconstruct my dendrogram with my specified color dictionary or (ii) reformat my D_leaf_color dictionary for the link_color_func parameter.
```
# Init
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()

# Load data
from sklearn.datasets import load_diabetes

# Clustering
from scipy.cluster.hierarchy import dendrogram, fcluster, leaves_list
from scipy.spatial import distance
from fastcluster import linkage # You can use SciPy one too

%matplotlib inline

# Dataset
A_data = load_diabetes().data
DF_diabetes = pd.DataFrame(A_data, columns = ["attr_%d" % j for j in range(A_data.shape[1])])

# Absolute value of correlation matrix, then subtract from 1 for disimilarity
DF_dism = 1 - np.abs(DF_diabetes.corr())

# Compute average linkage
A_dist = distance.squareform(DF_dism.as_matrix())
Z = linkage(A_dist,method="average")

# Color mapping
D_leaf_colors = {"attr_1": "#808080", # Unclustered gray

                 "attr_4": "#B061FF", # Cluster 1 indigo
                 "attr_5": "#B061FF",
                 "attr_2": "#B061FF",
                 "attr_8": "#B061FF",
                 "attr_6": "#B061FF",
                 "attr_7": "#B061FF",

                 "attr_0": "#61ffff", # Cluster 2 cyan
                 "attr_3": "#61ffff",
                 "attr_9": "#61ffff",
                 }

# Dendrogram
# To get this dendrogram coloring below  `color_threshold=0.7`
D = dendrogram(Z=Z, labels=DF_dism.index, color_threshold=None, leaf_font_size=12, leaf_rotation=45, link_color_func=D_leaf_colors)
# TypeError: 'dict' object is not callable
```
I also tried how do I get the subtrees of dendrogram made by scipy.cluster.hierarchy

Response:
Here a solution that uses the return matrix Z of linkage() (described early but a little hidden in the docs) and link_color_func:
```
# see question for code prior to "color mapping"

# Color mapping
dflt_col = "#808080"   # Unclustered gray
D_leaf_colors = {"attr_1": dflt_col,

                 "attr_4": "#B061FF", # Cluster 1 indigo
                 "attr_5": "#B061FF",
                 "attr_2": "#B061FF",
                 "attr_8": "#B061FF",
                 "attr_6": "#B061FF",
                 "attr_7": "#B061FF",

                 "attr_0": "#61ffff", # Cluster 2 cyan
                 "attr_3": "#61ffff",
                 "attr_9": "#61ffff",
                 }

# notes:
# * rows in Z correspond to "inverted U" links that connect clusters
# * rows are ordered by increasing distance
# * if the colors of the connected clusters match, use that color for link
link_cols = {}
for i, i12 in enumerate(Z[:,:2].astype(int)):
  c1, c2 = (link_cols[x] if x > len(Z) else D_leaf_colors["attr_%d"%x]
    for x in i12)
  link_cols[i+1+len(Z)] = c1 if c1 == c2 else dflt_col

# Dendrogram
D = dendrogram(Z=Z, labels=DF_dism.index, color_threshold=None,
  leaf_font_size=12, leaf_rotation=45, link_color_func=lambda x: link_cols[x])
```
Here the output:


Question: SciPy: leastsq vs least_squares SciPy provides two functions for nonlinear least squares problems: optimize.leastsq() uses the Levenberg-Marquardt algorithm only. optimize.least_squares() allows us to choose the Levenberg-Marquardt, Trust Region Reflective, or Trust Region Dogleg algorithm. Should we always use least_squares() instead of leastsq()? If so, what purpose does the latter serve?

Response: Short answer Should we always use least_squares() instead of leastsq()? Yes. If so, what purpose does the latter serve? Backward compatibility. Explanation The least_squares function is new in 0.17.1. Its documentation refers to leastsq as A legacy wrapper for the MINPACK implementation of the Levenberg-Marquadt algorithm. The original commit introducing least_squares actually called leastsq when the method was chosen to be 'lm'. But the contributor (Nikolay Mayorov) then decided that least_squares might feel more solid and homogeneous if I write a new wrapper to MINPACK functions, instead of calling leastsq. and so he did. So, leastsq is no longer required by least_squares, but I'd expect it to be kept at least for a while, to avoid breaking old code.


Question: How to find all neighbors of a given point in a delaunay triangulation using scipy.spatial.Delaunay? I have been searching for an answer to this question but cannot find anything useful. I am working with the python scientific computing stack (scipy,numpy,matplotlib) and I have a set of 2 dimensional points, for which I compute the Delaunay traingulation (wiki) using scipy.spatial.Delaunay. I need to write a function that, given any point a, will return all other points which are vertices of any simplex (i.e. triangle) that a is also a vertex of (the neighbors of a in the triangulation). However, the documentation for scipy.spatial.Delaunay (here) is pretty bad, and I can't for the life of me understand how the simplices are being specified or I would go about doing this. Even just an explanation of how the neighbors, vertices and vertex_to_simplex arrays in the Delaunay output are organized would be enough to get me going. Much thanks for any help.

Response:
I figured it out on my own, so here's an explanation for anyone future person who is confused by this. As an example, let's use the simple lattice of points that I was working with in my code, which I generate as follows
```
import numpy as np
import itertools as it
from matplotlib import pyplot as plt
import scipy as sp

inputs = list(it.product([0,1,2],[0,1,2]))
i = 0
lattice = range(0,len(inputs))
for pair in inputs:
    lattice[i] = mksite(pair[0], pair[1])
    i = i +1
```
Details here not really important, suffice to say it generates a regular triangular lattice in which the distance between a point and any of its six nearest neighbors is 1. To plot it
```
plt.plot(*np.transpose(lattice), marker = 'o', ls = '')
axes().set_aspect('equal')
```
Now compute the triangulation:
```
dela = sp.spatial.Delaunay
triang = dela(lattice)
```
Let's look at what this gives us.
```
triang.points
```
output:
```
array([[ 0.        ,  0.        ],
       [ 0.5       ,  0.8660254 ],
       [ 1.        ,  1.73205081],
       [ 1.        ,  0.        ],
       [ 1.5       ,  0.8660254 ],
       [ 2.        ,  1.73205081],
       [ 2.        ,  0.        ],
       [ 2.5       ,  0.8660254 ],
       [ 3.        ,  1.73205081]])
```
simple, just an array of all nine points in the lattice illustrated above. How let's look at:
```
triang.vertices
```
output:
```
array([[4, 3, 6],
       [5, 4, 2],
       [1, 3, 0],
       [1, 4, 2],
       [1, 4, 3],
       [7, 4, 6],
       [7, 5, 8],
       [7, 5, 4]], dtype=int32)
```
In this array, each row represents one simplex (triangle) in the triangulation. The three entries in each row are the indices of the vertices of that simplex in the points array we just saw. So for example the first simplex in this array, [4, 3, 6] is composed of the points:
```
[ 1.5       ,  0.8660254 ]
[ 1.        ,  0.        ]
[ 2.        ,  0.        ]
```
Its easy to see this by drawing the lattice on a piece of paper, labeling each point according to its index, and then tracing through each row in triang.vertices. This is all the information we need to write the function I specified in my question. It looks like:
```
def find_neighbors(pindex, triang):
    neighbors = list()
    for simplex in triang.vertices:
        if pindex in simplex:
            neighbors.extend([simplex[i] for i in range(len(simplex)) if simplex[i] != pindex])
            '''
            this is a one liner for if a simplex contains the point we`re interested in,
            extend the neighbors list by appending all the *other* point indices in the simplex
            '''
    #now we just have to strip out all the dulicate indices and return the neighbors list:
    return list(set(neighbors))
```
And that's it! I'm sure the function above could do with some optimization, its just what I came up with in a few minutes. If anyone has any suggestions, feel free to post them. Hopefully this helps somebody in the future who is as confused about this as I was.


Question:
Python: Finding multiple roots of nonlinear equation Assume the following function: f(x) = x * cos(x-4) With x = [-2.5, 2.5] this function crosses 0 at f(0) = 0 and f(-0.71238898) = 0. This was determined with the following code:
```
import math
from scipy.optimize import fsolve
def func(x):
    return x*math.cos(x-4)
x0 = fsolve(func, 0.0)
# returns [0.]
x0 = fsolve(func, -0.75)
# returns [-0.71238898]
```
What is the proper way to use fzero (or any other Python root finder) to find both roots in one call? Is there a different scipy function that does this? fzero reference

Response:
I once wrote a module for this task. It's based on chapter 4.3 from the book Numerical Methods in Engineering with Python by Jaan Kiusalaas:
```
import math

def rootsearch(f,a,b,dx):
    x1 = a; f1 = f(a)
    x2 = a + dx; f2 = f(x2)
    while f1*f2 > 0.0:
        if x1 >= b:
            return None,None
        x1 = x2; f1 = f2
        x2 = x1 + dx; f2 = f(x2)
    return x1,x2

def bisect(f,x1,x2,switch=0,epsilon=1.0e-9):
    f1 = f(x1)
    if f1 == 0.0:
        return x1
    f2 = f(x2)
    if f2 == 0.0:
        return x2
    if f1*f2 > 0.0:
        print('Root is not bracketed')
        return None
    n = int(math.ceil(math.log(abs(x2 - x1)/epsilon)/math.log(2.0)))
    for i in range(n):
        x3 = 0.5*(x1 + x2); f3 = f(x3)
        if (switch == 1) and (abs(f3) >abs(f1)) and (abs(f3) > abs(f2)):
            return None
        if f3 == 0.0:
            return x3
        if f2*f3 < 0.0:
            x1 = x3
            f1 = f3
        else:
            x2 =x3
            f2 = f3
    return (x1 + x2)/2.0

def roots(f, a, b, eps=1e-6):
    print ('The roots on the interval [%f, %f] are:' % (a,b))
    while 1:
        x1,x2 = rootsearch(f,a,b,eps)
        if x1 != None:
            a = x2
            root = bisect(f,x1,x2,1)
            if root != None:
                pass
                print (round(root,-int(math.log(eps, 10))))
        else:
            print ('\nDone')
            break

f=lambda x:x*math.cos(x-4)
roots(f, -3, 3)
```
roots finds all roots of f in the interval [a, b].


Question:
Parallel optimizations in SciPy I have a simple function
```
def square(x, a=1):
    return [x**2 + a, 2*x]
```
I want to minimize it over x, for several parameters a. I currently have loops that, in spirit, do something like this:
```

```python
from scipy import optimize
```

```python
res = optimize.minimize(square, 25, method='BFGS', jac=True)
```

```python
[res.x, res.fun]
#Output
#[array([ 0.]), 1.0]
```

```python
l = lambda x: square(x, 2)
```

```python
res = optimize.minimize(l, 25, method='BFGS', jac=True)
```

```python
[res.x, res.fun]
#Output
#[array([ 0.]), 2.0]
#```
#Now, the function is already vectorized
#```
```

```python
square(array([2,3]))
#Output
#[array([ 5, 10]), array([4, 6])]
```

```python
square(array([2,3]), array([2,3]))
#Output
#[array([ 6, 12]), array([4, 6])]
#```
#Which means it would probably be much faster to run all the optimizations in parallel rather than looping. Is that something that's easily do-able with SciPy? Or any other 3rd party tool?
```

Response:
Here's another try, based on my original answer and the discussion that followed. As far as I know, the scipy.optimize module is for functions with scalar or vector inputs and a scalar output, or "cost". Since you're treating each equation as independent of the others, my best idea is to use the multiprocessing module to do the work in parallel. If the functions you're minimizing are as simple as the ones in your question, I'd say it's not worth the effort. If the functions are more complex, and you'd like to divide the work up, try something like:
```
import numpy as np
from scipy import optimize
from multiprocessing import Pool

def square(x, a=1):
    return [np.sum(x**2 + a), 2*x]

def minimize(args):
    f,x,a = args
    res = optimize.minimize(f, x, method = 'BFGS', jac = True, args = [a])
    return res.x

# your a values
a = np.arange(1,11)

# initial guess for all the x values
x = np.empty(len(a))
x[:] = 25

args = [(square,a[i],x[i]) for i in range(10)]
p = Pool(4)
print p.map(minimize,args)
```


Question:
Slicing arrays in Numpy / Scipy I have an array like:
```
a = array([[1,2,3],[3,4,5],[4,5,6]])
```
What's the most efficient way to slice out a 1x2 array out of this that has only the first two columns of "a"? i.e.
```
array([[2,3],[4,5],[5,6]]) in this case.
```

Response:
Two dimensional numpy arrays are indexed using a[i,j] (not a[i][j]), but you can use the same slicing notation with numpy arrays and matrices as you can with ordinary matrices in python (just put them in a single []):
```

```python
from numpy import array
```

```python
a = array([[1,2,3],[3,4,5],[4,5,6]])
```

```python
a[:,1:]
#Output
#array([[2, 3],
#       [4, 5],
#       [5, 6]])
#```
```


Question: How to add clipboard support to Matplotlib figures? In MATLAB, there is a very convenient option to copy the current figure to the clipboard. Although Python/numpy/scipy/matplotlib is a great alternative to MATLAB, such an option is unfortunately missing. Can this option easily be added to Matplotlib figures? Preferably, all MPL figures should automatically benefit from this functionality. I'm using MPL's Qt4Agg backend, with PySide.

Response:
Yes, it can. The idea is to replace the default plt.figure with a custom one (a technique known as monkey patching) that injects a keyboard handler for copying to the clipboard. The following code will allow you to copy any MPL figure to the clipboard by pressing Ctrl+C:
```
import io
import matplotlib.pyplot as plt
from PySide.QtGui import QApplication, QImage

def add_clipboard_to_figures():
    # use monkey-patching to replace the original plt.figure() function with
    # our own, which supports clipboard-copying
    oldfig = plt.figure

    def newfig(*args, **kwargs):
        fig = oldfig(*args, **kwargs)
        def clipboard_handler(event):
            if event.key == 'ctrl+c':
                # store the image in a buffer using savefig(), this has the
                # advantage of applying all the default savefig parameters
                # such as background color; those would be ignored if you simply
                # grab the canvas using Qt
                buf = io.BytesIO()
                fig.savefig(buf)
                QApplication.clipboard().setImage(QImage.fromData(buf.getvalue()))
                buf.close()

        fig.canvas.mpl_connect('key_press_event', clipboard_handler)
        return fig

    plt.figure = newfig

add_clipboard_to_figures()
```
Note that if you want to use from matplotlib.pyplot import * (e.g. in an interactive session), you need to do so after you've executed the above code, otherwise the figure you import into the default namespace will be the unpatched version.


Question:
How to get confidence intervals from curve_fit My question involves statistics and python and I am a beginner in both. I am running a simulation, and for each value for the independent variable (X) I produce 1000 values for the dependent variable (Y). What I have done is that I calculated the average of Y for each value of X and fitted these averages using scipy.optimize.curve_fit. The curve fits nicely, but I want to draw also the confidence intervals. I am not sure if what I am doing is correct or if what I want to do can be done, but my question is how can I get the confidence intervals from the covariance matrix produced by curve_fit. The code reads the averages from files first then it just simply uses curve_fit.
```
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

def readTDvsTx(L, B, P, fileformat):
    # L should be '_Fixed_' or '_'
    TD = []
    infile = open(fileformat.format(L, B, P), 'r')
    infile.readline()  # To remove header
    for line in infile:
        l = line.split()  # each line contains TxR followed by CD followed by TD
        if eval(l[0]) >= 70 and eval(l[0]) <=190:
            td = eval(l[2])
            TD.append(td)
    infile.close()
    tdArray = np.array(TD)

    return tdArray

def rec(x, a, b):
    return a * (1 / (x**2)) + b

fileformat = 'Densities_file{}BS{}_PRNTS{}.txt'
txR = np.array(range(70, 200, 20))
parents = np.array(range(1,6))
disc_p1 = readTDvsTx('_Fixed_', 5, 1, fileformat)

popt, pcov = curve_fit(rec, txR, disc_p1)

plt.plot(txR, rec(txR, popt[0], popt[1]), 'r-')
plt.plot(txR, disc_p1, '.')

print(popt)
plt.show()
```
And here is the resulting fit:

Response:
Here's a quick and wrong answer: you can approximate the errors from the covariance matrix for your a and b parameters as the square root of its diagonals: np.sqrt(np.diagonal(pcov)). The parameter uncertainties can then be used to draw the confidence intervals. The answer is wrong because you before you fit your data to a model, you'll need an estimate of the errors on your averaged disc_p1 points. When averaging, you have lost the information about the scatter of the population, leading curve_fit to believe that the y-points you feed it are absolute and undisputable. This might cause an underestimation of your parameter errors. For an estimate of the uncertainties of your averaged Y values, you need to estimate their dispersion measure and pass it along to curve_fit while saying that your errors are absolute. Below is an example of how to do this for a random dataset where each of your points consists of a 1000 samples drawn from a normal distribution.
```
from scipy.optimize import curve_fit
import matplotlib.pylab as plt
import numpy as np

# model function
func = lambda x, a, b: a * (1 / (x**2)) + b

# approximating OP points
n_ypoints = 7
x_data = np.linspace(70, 190, n_ypoints)

# approximating the original scatter in Y-data
n_nested_points = 1000
point_errors = 50
y_data = [func(x, 4e6, -100) + np.random.normal(x, point_errors,
          n_nested_points) for x in x_data]

# averages and dispersion of data
y_means = np.array(y_data).mean(axis = 1)
y_spread = np.array(y_data).std(axis = 1)

best_fit_ab, covar = curve_fit(func, x_data, y_means,
                               sigma = y_spread,
                               absolute_sigma = True)
sigma_ab = np.sqrt(np.diagonal(covar))

from uncertainties import ufloat
a = ufloat(best_fit_ab[0], sigma_ab[0])
b = ufloat(best_fit_ab[1], sigma_ab[1])
text_res = "Best fit parameters:\na = {}\nb = {}".format(a, b)
print(text_res)

# plotting the unaveraged data
flier_kwargs = dict(marker = 'o', markerfacecolor = 'silver',
                    markersize = 3, alpha=0.7)
line_kwargs = dict(color = 'k', linewidth = 1)
bp = plt.boxplot(y_data, positions = x_data,
                 capprops = line_kwargs,
                 boxprops = line_kwargs,
                 whiskerprops = line_kwargs,
                 medianprops = line_kwargs,
                 flierprops = flier_kwargs,
                 widths = 5,
                 manage_ticks = False)
# plotting the averaged data with calculated dispersion
#plt.scatter(x_data, y_means, facecolor = 'silver', alpha = 1)
#plt.errorbar(x_data, y_means, y_spread, fmt = 'none', ecolor = 'black')

# plotting the model
hires_x = np.linspace(50, 190, 100)
plt.plot(hires_x, func(hires_x, *best_fit_ab), 'black')
bound_upper = func(hires_x, *(best_fit_ab + sigma_ab))
bound_lower = func(hires_x, *(best_fit_ab - sigma_ab))
# plotting the confidence intervals
plt.fill_between(hires_x, bound_lower, bound_upper,
                 color = 'black', alpha = 0.15)
plt.text(140, 800, text_res)
plt.xlim(40, 200)
plt.ylim(0, 1000)
plt.show()
```
Edit: If you are not considering the intrinsic errors on the data points, you are probably fine with using the "qiuck and wrong" case I mentioned before. The square root of the diagonal entries of covariance matrix can then be used to calculate your confidence intervals. However, note that the confidence intervals have shrunk now that we've dropped the uncertainties:
```
from scipy.optimize import curve_fit
import matplotlib.pylab as plt
import numpy as np

func = lambda x, a, b: a * (1 / (x**2)) + b

n_ypoints = 7
x_data = np.linspace(70, 190, n_ypoints)

y_data = np.array([786.31, 487.27, 341.78, 265.49,
                    224.76, 208.04, 200.22])
best_fit_ab, covar = curve_fit(func, x_data, y_data)
sigma_ab = np.sqrt(np.diagonal(covar))

# an easy way to properly format parameter errors
from uncertainties import ufloat
a = ufloat(best_fit_ab[0], sigma_ab[0])
b = ufloat(best_fit_ab[1], sigma_ab[1])
text_res = "Best fit parameters:\na = {}\nb = {}".format(a, b)
print(text_res)

plt.scatter(x_data, y_data, facecolor = 'silver',
            edgecolor = 'k', s = 10, alpha = 1)

# plotting the model
hires_x = np.linspace(50, 200, 100)
plt.plot(hires_x, func(hires_x, *best_fit_ab), 'black')
bound_upper = func(hires_x, *(best_fit_ab + sigma_ab))
bound_lower = func(hires_x, *(best_fit_ab - sigma_ab))
# plotting the confidence intervals
plt.fill_between(hires_x, bound_lower, bound_upper,
                 color = 'black', alpha = 0.15)
plt.text(140, 630, text_res)
plt.xlim(60, 200)
plt.ylim(0, 800)
plt.show()
```
If you're unsure whether to include the absolute errors or how to estimate them in your case, you'd be better off asking for advice at Cross Validated, as Stack Overflow is mainly for discussion on implementations of regression methods and not for discussion on the underlying statistics.