{"id":4340,"date":"2026-08-27T18:35:00","date_gmt":"2026-08-27T18:35:00","guid":{"rendered":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/"},"modified":"2026-08-28T08:59:08","modified_gmt":"2026-08-28T08:59:08","slug":"the-sigmoid-function-from-e-to-neural-networks","status":"publish","type":"post","link":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/","title":{"rendered":"The Sigmoid Perform: From ‘e’ to Neural Networks"},"content":{"rendered":"


\n<\/p>\n

\n

Welcome again!<\/p>\n

We lately mentioned backpropagation, and I hope you now have an concept of what backpropagation is and the way it really works.<\/p>\n

Let’s proceed the deep studying journey.<\/p>\n

Despite the fact that we apply the backpropagation algorithm to a neural community, we nonetheless have some issues, and vanishing gradients is considered one of them.<\/p>\n

Whereas I used to be studying about vanishing gradients, I got here throughout the sigmoid perform.<\/p>\n

Everyone knows that it’s utilized in logistic regression, the place we apply the sigmoid perform to a worth to acquire an output between 0 and 1.<\/p>\n

Now, right here in neural networks, it may be used as an activation perform.<\/p>\n

What I learn about sigmoid is the equation we’ve got and its utilization in logistic regression and neural networks.<\/p>\n

I used to be inquisitive about how we get this equation and the story behind it.<\/p>\n

On this weblog, let’s examine how we get to the sigmoid equation.<\/p>\n

By the way in which, if you have not learn Half 3 of the backpropagation sequence, you’ll be able to learn it right here.<\/p>\n

\u00b7\u00b7\u00b7<\/p>\n

How Do We Really Use Sigmoid?<\/h3>\n

We already know the equation of the sigmoid perform.<\/p>\n

\u03c3(x)=11+e\u2212xsigma(x) = frac{1}{1 + e^{-x}}<\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Earlier than we proceed, let’s examine how we use it in logistic regression.<\/p>\n

For instance, we wish to predict whether or not a scholar will cross or fail primarily based on the variety of hours they studied.<\/p>\n

We’re utilizing the logistic regression mannequin right here.<\/p>\n

First, it calculates a rating<\/p>\n

For instance the rating for a scholar is:<\/p>\n

This rating will not be a likelihood. <\/p>\n

It’s simply the linear mixture of parameters.<\/p>\n

Now we cross it via the sigmoid perform:<\/p>\n

\u03c3(z)=11+e\u2212zsigma(z) = frac{1}{1 + e^{-z}}<\/span>\u03c3<\/span>(<\/span>z<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>z<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

we get,<\/p>\n

\u03c3(2)=11+e\u22122\u22480.88sigma(2) = frac{1}{1 + e^{-2}} approx 0.88<\/span>\u03c3<\/span>(<\/span>2<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>0.88<\/span><\/span><\/span><\/span><\/span><\/div>\n

The sigmoid perform all the time produces an output between 0 and 1.<\/p>\n

Right here the output is roughly 0.88 or 88%.<\/p>\n

In logistic regression, this may be interpreted as an 88% likelihood of the coed passing the examination.<\/p>\n

We will then use a threshold, comparable to 0.5, to make the ultimate classification.<\/p>\n

In brief, the movement will be like<\/p>\n

Rating\u2192Sigmoid\u2192Likelihood\u2192Classtext{Rating} rightarrow textual content{Sigmoid} rightarrow textual content{Likelihood} rightarrow textual content{Class}<\/span>Rating<\/span><\/span>\u2192<\/span><\/span>Sigmoid<\/span><\/span>\u2192<\/span><\/span>Likelihood<\/span><\/span>\u2192<\/span><\/span>Class<\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

That is how we generally use the sigmoid perform in logistic regression.<\/p>\n

However What Is This \u201ce\u201d?<\/h3>\n

Now, let’s as soon as once more have a look at the sigmoid equation.<\/p>\n

\u03c3(z)=11+e\u2212zsigma(z) = frac{1}{1 + e^{-z}}<\/span>\u03c3<\/span>(<\/span>z<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>z<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

The very first thing we discover is the e.<\/p>\n

We all know that it’s a mathematical fixed and its worth is<\/p>\n

e\u22482.71828e approx 2.71828<\/span>e<\/span>\u2248<\/span><\/span>2.71828<\/span><\/span><\/span><\/span><\/span><\/div>\n

However what precisely is ‘e’?<\/p>\n

Why is that this quantity current within the sigmoid equation?<\/p>\n

Let’s take a step again and perceive the place this quantity comes from.<\/p>\n

One factor is that right here we’re not making an attempt to find ‘e’, however the aim is to know the importance of ‘e’ and see the place it naturally seems.<\/p>\n

Now let’s go to the financial institution and see what we will observe.<\/p>\n

Let’s Begin with a Easy Financial institution Instance<\/h3>\n

Think about we deposited Rs.100 right into a checking account.<\/p>\n

For instance the financial institution is giving us a 100% annual rate of interest.<\/p>\n

If the financial institution provides all the yr’s curiosity on the finish of the yr, we earn Rs.100 in curiosity.<\/p>\n

So after one yr, we’ve got<\/p>\n

100+100=200100 + 100 = 200<\/span>100<\/span>+<\/span><\/span>100<\/span>=<\/span><\/span>200<\/span><\/span><\/span><\/span><\/span><\/div>\n

We will additionally write it as<\/p>\n

100(1+1)=200100(1 + 1) = 200<\/span>100<\/span>(<\/span>1<\/span>+<\/span><\/span>1<\/span>)<\/span>=<\/span><\/span>200<\/span><\/span><\/span><\/span><\/span><\/div>\n

Rs.100 turned Rs.200 after one yr.<\/p>\n

However now let’s change one factor.<\/p>\n

What if the financial institution would not wait till the tip of the yr so as to add the curiosity?<\/p>\n

What if it provides the curiosity twice a yr?<\/p>\n

The annual rate of interest continues to be 100%.<\/p>\n

However now the yr is split into two intervals.<\/p>\n

So for every six-month interval we get half of the annual rate of interest:<\/p>\n

12=0.5=50percentfrac{1}{2} = 0.5 = 50%<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.5<\/span>=<\/span><\/span>50%<\/span><\/span><\/span><\/span><\/span><\/div>\n

Through the first six months, we get<\/p>\n

100(1+12)=150100left(1 + frac{1}{2}proper) = 150<\/span>100<\/span>(<\/span><\/span>1<\/span>+<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>=<\/span><\/span>150<\/span><\/span><\/span><\/span><\/span><\/div>\n

After six months, we’ve got Rs.150.<\/p>\n

Through the subsequent six months, the curiosity is calculated on this new quantity<\/p>\n

150(1+12)=225150left(1 + frac{1}{2}proper) = 225<\/span>150<\/span>(<\/span><\/span>1<\/span>+<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>=<\/span><\/span>225<\/span><\/span><\/span><\/span><\/span><\/div>\n

Then we’ve got<\/p>\n

100(1+12)2=225100left(1 + frac{1}{2}proper)^2 = 225<\/span>100<\/span>(<\/span><\/span>1<\/span>+<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>225<\/span><\/span><\/span><\/span><\/span><\/div>\n

Why did we get Rs.225 as a substitute of Rs.200?<\/p>\n

As a result of the curiosity earned in the course of the first six months additionally earned curiosity in the course of the second six months.<\/p>\n

In easy phrases we will say<\/p>\n

‘curiosity earns curiosity’<\/p>\n

That is the essential concept behind compound curiosity.<\/p>\n

What Occurs When We Compound Extra Steadily?<\/h4>\n

Now let’s make the compounding extra frequent.<\/p>\n

If we compound 4 instances a yr:<\/p>\n

100(1+14)4\u2248244.14100left(1 + frac{1}{4}proper)^4
\napprox 244.14<\/span>100<\/span>(<\/span><\/span>1<\/span>+<\/span>4<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>4<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>244.14<\/span><\/span><\/span><\/span><\/span><\/div>\n

If we compound 12 instances a yr:<\/p>\n

100(1+112)12\u2248261.30100left(1 + frac{1}{12}proper)^{12}
\napprox 261.30<\/span>100<\/span>(<\/span><\/span>1<\/span>+<\/span>12<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>12<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>261.30<\/span><\/span><\/span><\/span><\/span><\/div>\n

If we compound day-after-day:<\/p>\n

100(1+1365)365\u2248271.46100left(1 + frac{1}{365}proper)^{365}
\napprox 271.46<\/span>100<\/span>(<\/span><\/span>1<\/span>+<\/span>365<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>365<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>271.46<\/span><\/span><\/span><\/span><\/span><\/div>\n

Observe the sample.<\/p>\n

As we enhance the variety of compounding intervals, the ultimate quantity retains rising.<\/p>\n

The reason being that development is being utilized repeatedly to an quantity that has already elevated.<\/p>\n

The place Does e Come From?<\/h3>\n

The Rs.100 will not be the vital half right here.<\/p>\n

Let’s take away it and have a look at the expansion issue:<\/p>\n

(1+1n)nleft(1 + frac{1}{n}proper)^n<\/span>(<\/span><\/span>1<\/span>+<\/span>n<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>n<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Right here, ‘n’ represents the variety of instances we compound in the course of the yr.<\/p>\n

For instance:<\/p>\n

(1+11)1=2left(1 + frac{1}{1}proper)^1 = 2<\/span>(<\/span><\/span>1<\/span>+<\/span>1<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/div>\n
(1+12)2=2.25left(1 + frac{1}{2}proper)^2 = 2.25<\/span>(<\/span><\/span>1<\/span>+<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>2.25<\/span><\/span><\/span><\/span><\/span><\/div>\n
(1+14)4\u22482.4414left(1 + frac{1}{4}proper)^4 approx 2.4414<\/span>(<\/span><\/span>1<\/span>+<\/span>4<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>4<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>2.4414<\/span><\/span><\/span><\/span><\/span><\/div>\n
(1+112)12\u22482.613left(1 + frac{1}{12}proper)^{12} approx 2.613<\/span>(<\/span><\/span>1<\/span>+<\/span>12<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>12<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>2.613<\/span><\/span><\/span><\/span><\/span><\/div>\n
(1+1365)365\u22482.7146left(1 + frac{1}{365}proper)^{365} approx 2.7146<\/span>(<\/span><\/span>1<\/span>+<\/span>365<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>365<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>2.7146<\/span><\/span><\/span><\/span><\/span><\/div>\n

As we make the compounding increasingly more frequent, the worth will get nearer and nearer to<\/p>\n

2.71828\u20262.71828ldots<\/span>2.71828<\/span>\u2026<\/span><\/span><\/span><\/span><\/span><\/div>\n

This quantity is named ‘e’<\/p>\n

e\u22482.71828e approx 2.71828<\/span>e<\/span>\u2248<\/span><\/span>2.71828<\/span><\/span><\/span><\/span><\/span><\/div>\n

Mathematically, we will specific this concept utilizing a restrict<\/p>\n

e=lim\u2061n\u2192\u221e(1+1n)ne = lim_{n rightarrow infty}
\nleft(1 + frac{1}{n}proper)^n<\/span>e<\/span>=<\/span><\/span>n<\/span>\u2192<\/span>\u221e<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>(<\/span><\/span>1<\/span>+<\/span>n<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>n<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

The notation could look advanced, however the concept is easy.<\/p>\n

Right here, we’re asking:<\/p>\n

“What worth does this expression method as ‘n’ turns into bigger and bigger?”<\/p>\n

As ‘n’ will increase:<\/p>\n

(1+1n)nleft(1 + frac{1}{n}proper)^n<\/span>(<\/span><\/span>1<\/span>+<\/span>n<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span>n<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

will get nearer and nearer to:<\/p>\n

2.71828\u20262.71828ldots<\/span>2.71828<\/span>\u2026<\/span><\/span><\/span><\/span><\/span><\/div>\n

That limiting worth is ‘e’.<\/p>\n

So, What Does the Financial institution Need to Do with Sigmoid?<\/h3>\n

However why are we speaking about this and what does this checking account should do with sigmoid.<\/p>\n

This instance is not to clarify compound curiosity, however it provides us an instinct for the place ‘e’ naturally seems.<\/p>\n

The vital concept right here is repeated development.<\/p>\n

When development is repeatedly utilized to an quantity that has already grown, we get a compounding course of.<\/p>\n

And when that course of occurs repeatedly extra incessantly, the quantity ‘e’ naturally seems.<\/p>\n

So as a substitute of merely memorizing that<\/p>\n

e\u22482.71828e approx 2.71828<\/span>e<\/span>\u2248<\/span><\/span>2.71828<\/span><\/span><\/span><\/span><\/span><\/div>\n

we now have some instinct behind it.<\/p>\n

The Particular Property of e<\/h3>\n

From the financial institution instance, we noticed that ‘e’ naturally seems after we have a look at repeated development and steady compounding.<\/p>\n

However ‘e’ is greater than only a quantity that seems in compound curiosity.<\/p>\n

It has a really particular property after we have a look at it via calculus.<\/p>\n

Let’s think about the exponential perform<\/p>\n

If we differentiate this perform, we get<\/p>\n

dydx=exfrac{dy}{dx}=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

This formulation we already know.<\/p>\n

However what does the by-product inform us?<\/p>\n

We already know that it tells us the speed of change of a perform.<\/p>\n

For instance, if we’ve got<\/p>\n

its by-product is<\/p>\n

dydx=2xfrac{dy}{dx}=2x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>2<\/span>x<\/span><\/span><\/span><\/span><\/span><\/div>\n

Which means that the speed at which x2 modifications is determined by the worth of x.<\/p>\n

At x=1:<\/p>\n

dydx=2(1)=2frac{dy}{dx}=2(1)=2<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>2<\/span>(<\/span>1<\/span>)<\/span>=<\/span><\/span>2<\/span><\/span><\/span><\/span><\/span><\/div>\n

At x=3:<\/p>\n

dydx=2(3)=6frac{dy}{dx}=2(3)=6<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>2<\/span>(<\/span>3<\/span>)<\/span>=<\/span><\/span>6<\/span><\/span><\/span><\/span><\/span><\/div>\n

So, for x2, the perform and its charge of change are completely different.<\/p>\n

Now let us take a look at ex.<\/p>\n

For<\/p>\n

we’ve got<\/p>\n

dydx=exfrac{dy}{dx}=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Which means that the speed of change of ex is the same as its present worth.<\/p>\n

Let us take a look at some values.<\/p>\n

When x=0<\/p>\n

and<\/p>\n

dydx=1frac{dy}{dx}=1<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

When x=1<\/p>\n

e1\u22482.718e^1approx2.718<\/span>e<\/span>1<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>2.718<\/span><\/span><\/span><\/span><\/span><\/div>\n

and<\/p>\n

dydx\u22482.718frac{dy}{dx}approx2.718<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>2.718<\/span><\/span><\/span><\/span><\/span><\/div>\n

When x=2<\/p>\n

e2\u22487.389e^2approx7.389<\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>7.389<\/span><\/span><\/span><\/span><\/span><\/div>\n

and<\/p>\n

dydx\u22487.389frac{dy}{dx}approx7.389<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>7.389<\/span><\/span><\/span><\/span><\/span><\/div>\n

So, right here we will say that<\/p>\n

Charge of change = Present worth<\/p>\n

This is likely one of the most vital properties of the exponential perform with base e.<\/p>\n

Why Is the Spinoff of ex Equal to ex?<\/h3>\n

We now have an concept of an vital property of ‘e’ in calculus.<\/p>\n

ddxex=exfrac{d}{dx}e^x=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

We simply mentioned what it’s however let’s examine why does this occur?<\/p>\n

For those who already know why<\/p>\n

ddxex=exfrac{d}{dx}e^x=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

then use this part for fast revision as we join it again to the sigmoid perform.<\/p>\n

Beginning with a Common Exponential<\/h4>\n

First let’s think about a basic exponential perform.<\/p>\n

Right here, z is the bottom and x is the exponent.<\/p>\n

2x,3x,5x,10×2^x,qquad 3^x,qquad 5^x,qquad 10^x<\/span>2<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>,<\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>,<\/span>5<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>,<\/span>1<\/span>0<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

are all examples of this way.<\/p>\n

Now let’s examine what occurs after we differentiate zx<\/p>\n

We’ve got,<\/p>\n

dydx=lim\u2061h\u21920zx+h\u2212zxhfrac{dy}{dx}
\n=
\nlim_{hto0}
\nfrac{z^{x+h}-z^x}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>x<\/span>+<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Utilizing the exponent rule we get<\/p>\n

zx+h=zxzhz^{x+h}=z^xz^h<\/span>z<\/span>x<\/span>+<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Due to this fact<\/p>\n

dydx=lim\u2061h\u21920zxzh\u2212zxhfrac{dy}{dx}
\n=
\nlim_{hto0}
\nfrac{z^xz^h-z^x}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Now discover that zx seems in each phrases within the numerator.<\/p>\n

We will issue it out<\/p>\n

dydx=lim\u2061h\u21920zxzh\u22121hfrac{dy}{dx}
\n=
\nlim_{hto0}
\nz^xfrac{z^h-1}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Right here zx doesn’t rely on h, so we will take it exterior the restrict<\/p>\n

dydx=zxlim\u2061h\u21920zh\u22121hfrac{dy}{dx}
\n=
\nz^x
\nlim_{hto0}
\nfrac{z^h-1}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

And that is the place issues get attention-grabbing.<\/p>\n

Our result’s<\/p>\n

dydx=zxlim\u2061h\u21920zh\u22121hfrac{dy}{dx}
\n=
\nz^x
\nlim_{hto0}
\nfrac{z^h-1}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span>y<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Take a look at the 2 components individually.<\/p>\n

The primary half is<\/p>\n

That’s our unique exponential perform.<\/p>\n

The second half is<\/p>\n

lim\u2061h\u21920zh\u22121hlim_{hto0}
\nfrac{z^h-1}{h}<\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

We will see that there is no such thing as a ‘x’ on this expression.<\/p>\n

It is determined by the bottom ‘z’, however not on ‘x’.<\/p>\n

This implies, for any worth of ‘z’, this complete restrict is only a fixed.<\/p>\n

Let’s name this fixed ‘C’.<\/p>\n

C=lim\u2061h\u21920zh\u22121hC=
\nlim_{hto0}
\nfrac{z^h-1}{h}<\/span>C<\/span>=<\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Due to this fact we will write it as,<\/p>\n

ddxzx=Czxfrac{d}{dx}z^x=Cz^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>C<\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

This tells us one thing vital.<\/p>\n

Once we differentiate an exponential perform, we get the unique exponential perform, multiplied by a continuing.<\/p>\n

In different manner,<\/p>\n

Spinoff\u00a0of\u00a0zx=fixed\u00d7zxtext{Spinoff of }z^x
\n=
\ntextual content{fixed}instances z^x<\/span>Spinoff\u00a0of\u00a0<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>fixed<\/span><\/span>\u00d7<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

The Fixed Is determined by the Base<\/p>\n

Now let’s take an instance of exponential perform:<\/p>\n

From our outcome, we’ve got<\/p>\n

ddx3x=C3xfrac{d}{dx}3^x=C3^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>C<\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

For z=3, the fixed is<\/p>\n

C=lim\u2061h\u219203h\u22121hC=
\nlim_{hto0}
\nfrac{3^h-1}{h}<\/span>C<\/span>=<\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>3<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Now we have to discover the worth of this restrict.<\/p>\n

Let’s perceive this in intuitive manner.<\/p>\n

For the bottom 3, the worth of the fixed is roughly<\/p>\n

C\u22481.0986Capprox1.0986<\/span>C<\/span>\u2248<\/span><\/span>1.0986<\/span><\/span><\/span><\/span><\/span><\/div>\n

Due to this fact,<\/p>\n

ddx3x\u22481.0986(3x)frac{d}{dx}3^x
\napprox
\n1.0986(3^x)<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>1.0986<\/span>(<\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/span><\/div>\n

Let’s have a look at what this tells us through the use of at completely different ‘x’ values.<\/p>\n

When<\/p>\n

we’ve got<\/p>\n

the speed of change right here is roughly<\/p>\n

1.0986(1)=1.09861.0986(1)=1.0986<\/span>1.0986<\/span>(<\/span>1<\/span>)<\/span>=<\/span><\/span>1.0986<\/span><\/span><\/span><\/span><\/span><\/div>\n

When<\/p>\n

we get<\/p>\n

The speed of change is<\/p>\n

1.0986(3)\u22483.29581.0986(3)approx3.2958<\/span>1.0986<\/span>(<\/span>3<\/span>)<\/span>\u2248<\/span><\/span>3.2958<\/span><\/span><\/span><\/span><\/span><\/div>\n

And when<\/p>\n

we’ve got<\/p>\n

The speed of change is roughly<\/p>\n

1.0986(9)\u22489.88741.0986(9)approx9.8874<\/span>1.0986<\/span>(<\/span>9<\/span>)<\/span>\u2248<\/span><\/span>9.8874<\/span><\/span><\/span><\/span><\/span><\/div>\n

We will see that the by-product will not be precisely equal to 3x.<\/p>\n

As an alternative, we acquired<\/p>\n

ddx3x\u22481.0986(3x)frac{d}{dx}3^x
\napprox
\n1.0986(3^x)<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>1.0986<\/span>(<\/span>3<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/span><\/div>\n

The perform and its charge of change have the identical exponential form, however the charge of change is scaled by a continuing.<\/p>\n

Discovering the Particular Base<\/h3>\n

Now, we all know that<\/p>\n

ddxzx=Czxfrac{d}{dx}z^x=Cz^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>C<\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

The worth of ‘C’ trusted the bottom.<\/p>\n

For 3x,<\/p>\n

C\u22481.0986Capprox1.0986<\/span>C<\/span>\u2248<\/span><\/span>1.0986<\/span><\/span><\/span><\/span><\/span><\/div>\n

Okay however what if we might discover a base for which C is strictly 1?<\/p>\n

Do we’ve got any quantity? <\/p>\n

If sure, then we get<\/p>\n

Our by-product would turn into<\/p>\n

ddxzx=zxfrac{d}{dx}z^x=z^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

In different phrases, we will say that the perform can be precisely equal to its personal by-product.<\/p>\n

So, now we’re on the lookout for a base z that satisfies<\/p>\n

lim\u2061h\u21920zh\u22121h=1lim_{hto0}
\nfrac{z^h-1}{h}=1<\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

There’s one explicit optimistic quantity that satisfies this situation and also you all know what’s that quantity is.<\/p>\n

We name this quantity<\/p>\n

and its numerical worth is<\/p>\n

e\u22482.71828eapprox2.71828<\/span>e<\/span>\u2248<\/span><\/span>2.71828<\/span><\/span><\/span><\/span><\/span><\/div>\n

For this explicit base, the fixed turns into<\/p>\n

Due to this fact,<\/p>\n

ddxex=1\u22c5exfrac{d}{dx}e^x
\n=
\n1cdot e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>1<\/span>\u22c5<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

which supplies us<\/p>\n

ddxex=exfrac{d}{dx}e^x=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

\u00b7\u00b7\u00b7<\/p>\n

So What Did We Really Uncover?<\/h3>\n

We began with a basic exponential perform<\/p>\n

Utilizing the definition of a by-product, we discovered<\/p>\n

ddxzx=zxlim\u2061h\u21920zh\u22121hfrac{d}{dx}z^x
\n=
\nz^x
\nlim_{hto0}
\nfrac{z^h-1}{h}<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>h<\/span>\u2192<\/span>0<\/span><\/span><\/span><\/span>lim<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>h<\/span><\/span><\/span><\/span>z<\/span>h<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

We then noticed that the restrict is just a continuing that is determined by the bottom.<\/p>\n

Then we’ve got written it as<\/p>\n

ddxzx=Czxfrac{d}{dx}z^x=Cz^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>C<\/span>z<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Then we requested:<\/p>\n

Is there a base for which C=1?<\/p>\n

The reply is sure.<\/p>\n

That particular base is e.<\/p>\n

Due to this fact,<\/p>\n

ddxex=exfrac{d}{dx}e^x=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Now we’ve got an concept of how we acquired the by-product. <\/p>\n

Within the earlier financial institution instance, ‘e’ appeared via repeated development and steady compounding.<\/p>\n

Now, via calculus, we’ve got seen one other particular property of the identical quantity<\/p>\n

ddxex=exfrac{d}{dx}e^x=e^x<\/span>d<\/span>x<\/span><\/span><\/span><\/span>d<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

In easy phrases, we will say that ex grows at a charge equal to its present worth.<\/p>\n

Now, Let’s Return to Sigmoid<\/h3>\n

Let’s as soon as once more have a look at the sigmoid equation.<\/p>\n

\u03c3(x)=11+e\u2212xsigma(x)=frac{1}{1+e^{-x}}<\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Now we’ve got some concept of what ‘e’ really is.<\/p>\n

Now we deal with the entire equation.<\/p>\n

The query right here is why does the sigmoid perform is on this explicit type?<\/p>\n

To know this we must always return to logistic regression.<\/p>\n

We began with a uncooked rating<\/p>\n

‘z’ will be any actual quantity.<\/p>\n

However for classification, we needed to interpret the mannequin’s output as a likelihood.<\/p>\n

A likelihood should lie between 0 and 1<\/p>\n

0<p<1<\/span><\/p>\n

So we wish to rework any worth of ‘z’ into a worth between 0 and 1.<\/p>\n

In different phrases, we wish one thing that may obtain<\/p>\n

z\u2208(\u2212\u221e,\u221e)zin(-infty,infty)<\/span>z<\/span>\u2208<\/span><\/span>(<\/span>\u2212<\/span>\u221e<\/span>,<\/span>\u221e<\/span>)<\/span><\/span><\/span><\/span><\/span><\/div>\n

and produce:<\/p>\n

p\u2208(0,1)pin(0,1)<\/span>p<\/span>\u2208<\/span><\/span>(<\/span>0<\/span>,<\/span>1<\/span>)<\/span><\/span><\/span><\/span><\/span><\/div>\n

Constructing a Perform That Outputs Between 0 and 1<\/h3>\n

Now, the duty is to assemble such transformation.<\/p>\n

However how can we try this?<\/p>\n

Let’s begin with a quite simple statement.<\/p>\n

Suppose we’ve got a quantity higher than 1.<\/p>\n

For instance<\/p>\n

If we take its reciprocal, we get<\/p>\n

15=0.2frac{1}{5}=0.2<\/span>5<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.2<\/span><\/span><\/span><\/span><\/span><\/div>\n

which is between 0 and 1.<\/p>\n

The identical concept works for any numbers higher than 1<\/p>\n

12=0.5frac{1}{2}=0.5<\/span>2<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.5<\/span><\/span><\/span><\/span><\/span><\/div>\n
110=0.1frac{1}{10}=0.1<\/span>10<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.1<\/span><\/span><\/span><\/span><\/span><\/div>\n
1100=0.01frac{1}{100}=0.01<\/span>100<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.01<\/span><\/span><\/span><\/span><\/span><\/div>\n

Right here we will discover that<\/p>\n

If<\/p>\n

then<\/p>\n

0<1A<10 < frac{1}{A} < 1<\/span>0<\/span><<\/span><\/span>A<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><<\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

This provides us a easy concept.<\/p>\n

If we will have a amount that’s all the time higher than 1, then taking its reciprocal will mechanically give us a worth between 0 and 1.<\/p>\n

And that’s precisely the vary we wish for a likelihood.<\/p>\n

Nevertheless, there may be another factor we’d like.<\/p>\n

We don’t wish to use a hard and fast quantity comparable to 5 within the denominator.<\/p>\n

as a result of that all the time give us the identical output.<\/p>\n

Our output ought to change when the enter ‘x’ modifications.<\/p>\n

For instance, we wish a optimistic enter to supply a bigger likelihood, whereas a unfavourable enter ought to produce a smaller likelihood.<\/p>\n

So, we’d like a amount that modifications with x.<\/p>\n

Now e Enters the Image<\/h4>\n

You might be proper. It is time for ‘e’ to enter.<\/p>\n

That is the place the exponential perform we simply realized about turns into helpful.<\/p>\n

Exponential features are all the time optimistic, which suggests<\/p>\n

for each actual worth of x.<\/p>\n

For instance:<\/p>\n

e\u22122\u22480.1353e^{-2}approx0.1353<\/span>e<\/span>\u2212<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>0.1353<\/span><\/span><\/span><\/span><\/span><\/div>\n
e2\u22487.389e^2approx7.389<\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>7.389<\/span><\/span><\/span><\/span><\/span><\/div>\n

Whether or not the x is unfavourable, zero, or optimistic, ex by no means turns into unfavourable or zero.<\/p>\n

However the sigmoid equation comprises e-x. <\/p>\n

Until right here we solely mentioned about ex.<\/p>\n

So let’s first see what a unfavourable exponent means.<\/p>\n

We already know what a optimistic exponent means.<\/p>\n

For instance:<\/p>\n

e2=e\u00d7ee^2=etimes e<\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>\u00d7<\/span><\/span>e<\/span><\/span><\/span><\/span><\/span><\/div>\n

and:<\/p>\n

e3=e\u00d7e\u00d7ee^3=etimes etimes e<\/span>e<\/span>3<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>\u00d7<\/span><\/span>e<\/span>\u00d7<\/span><\/span>e<\/span><\/span><\/span><\/span><\/span><\/div>\n

A unfavourable exponent represents the reciprocal of the corresponding optimistic exponent.<\/p>\n

For instance:<\/p>\n

e\u22121=1ee^{-1}=frac{1}{e}<\/span>e<\/span>\u2212<\/span>1<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Equally<\/p>\n

e\u22122=1e2e^{-2}=frac{1}{e^2}<\/span>e<\/span>\u2212<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

and<\/p>\n

e\u22123=1e3e^{-3}=frac{1}{e^3}<\/span>e<\/span>\u2212<\/span>3<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>3<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

Normally, we will write as<\/p>\n

e\u2212x=1exe^{-x}=frac{1}{e^x}<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

So, e-x will not be a totally completely different perform.<\/p>\n

It’s merely the reciprocal of ex.<\/p>\n

Now we will use what we already learn about ex.<\/p>\n

Since:<\/p>\n

its reciprocal can also be optimistic<\/p>\n

1ex>0frac{1}{e^x}>0<\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>><\/span><\/span>0<\/span><\/span><\/span><\/span><\/span><\/div>\n

and since <\/p>\n

e\u2212x=1exe^{-x}=frac{1}{e^x}<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>e<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

we get<\/p>\n

for each actual worth of x.<\/p>\n

That is vital as a result of it provides us precisely the type of amount we’d like.<\/p>\n

If e-x is all the time optimistic, then including 1 provides us a amount that’s all the time higher than 1<\/p>\n

1+e\u2212x>11+e^{-x}>1<\/span>1<\/span>+<\/span><\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>><\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

And now we will use our reciprocal concept.<\/p>\n

If a quantity is bigger than 1, its reciprocal lies between 0 and 1<\/p>\n

0<11+e\u2212x<10<frac{1}{1+e^{-x}}<1<\/span>0<\/span><<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><<\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

Now we’ve got a perform whose output is all the time between 0 and 1.<\/p>\n

The expression we simply acquired is<\/p>\n

11+e\u2212xfrac{1}{1+e^{-x}}<\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

and that is precisely the sigmoid perform we began with<\/p>\n

\u03c3(x)=11+e\u2212xsigma(x)=frac{1}{1+e^{-x}}<\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

So as a substitute of trying on the sigmoid equation as a formulation, now we will perceive the instinct behind its construction.<\/p>\n

We needed the output to lie between 0 and 1.<\/p>\n

We noticed that the reciprocal of a quantity higher than 1 lies between 0 and 1.<\/p>\n

As e-x is all the time optimistic, we used it to assemble a amount higher than 1<\/p>\n

1+e\u2212x>11+e^{-x}>1<\/span>1<\/span>+<\/span><\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>><\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

Taking its reciprocal gave us<\/p>\n

\u03c3(x)=11+e\u2212xsigma(x)=frac{1}{1+e^{-x}}<\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

This gave us the vary we needed.<\/p>\n

However does this equation really behave the way in which we anticipated it to do?<\/p>\n

Right here, our aim is to know the instinct behind the construction of the sigmoid perform. <\/p>\n

There are different features that may map values to the vary 0 to 1, and why logistic regression makes use of sigmoid is said to odds and log-odds, a subject which we are going to discover in future blogs.<\/p>\n

Does the Sigmoid Behave the Manner We Anticipated?<\/h3>\n

Let’s take a look at on few values.<\/p>\n

First, let’s think about<\/p>\n

Substituting into the sigmoid perform:<\/p>\n

\u03c3(0)=11+e\u22120sigma(0)=frac{1}{1+e^{-0}}<\/span>\u03c3<\/span>(<\/span>0<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>0<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

as<\/p>\n

we get<\/p>\n

\u03c3(0)=11+1=0.5sigma(0)=frac{1}{1+1}=0.5<\/span>\u03c3<\/span>(<\/span>0<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>1<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>0.5<\/span><\/span><\/span><\/span><\/span><\/div>\n

When the enter is 0, the sigmoid provides us precisely 0.5.<\/p>\n

Now let’s take a optimistic quantity<\/p>\n

then<\/p>\n

\u03c3(2)=11+e\u22122sigma(2)=frac{1}{1+e^{-2}}<\/span>\u03c3<\/span>(<\/span>2<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

We already seen<\/p>\n

e\u22122\u22480.1353e^{-2}approx0.1353<\/span>e<\/span>\u2212<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>0.1353<\/span><\/span><\/span><\/span><\/span><\/div>\n

which supplies <\/p>\n

\u03c3(2)=11+0.1353=11.1353\u22480.881sigma(2)
\n=
\nfrac{1}{1+0.1353}
\n=
\nfrac{1}{1.1353}
\napprox 0.881<\/span>\u03c3<\/span>(<\/span>2<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>0.1353<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span><\/span>1.1353<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>0.881<\/span><\/span><\/span><\/span><\/span><\/div>\n

The sigmoid transformed the enter 2 into roughly 0.881 or 88.1%.<\/p>\n

Now let’s examine what occurs when the enter is a unfavourable quantity.<\/p>\n

Contemplate<\/p>\n

Then<\/p>\n

\u03c3(\u22122)=11+e\u2212(\u22122)sigma(-2)
\n=
\nfrac{1}{1+e^{-(-2)}}<\/span>\u03c3<\/span>(<\/span>\u2212<\/span>2<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>(<\/span>\u2212<\/span>2<\/span>)<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n
\u03c3(\u22122)=11+e2sigma(-2)
\n=
\nfrac{1}{1+e^2}<\/span>\u03c3<\/span>(<\/span>\u2212<\/span>2<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

We all know<\/p>\n

e2\u22487.389e^2approx7.389<\/span>e<\/span>2<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span><\/span>7.389<\/span><\/span><\/span><\/span><\/span><\/div>\n

Lastly we get<\/p>\n

\u03c3(\u22122)=11+7.389=18.389\u22480.119begin{aligned}
\nsigma(-2)
\n&=frac{1}{1+7.389}
\n&=frac{1}{8.389}
\n&approx0.119
\nfinish{aligned}<\/span>\u03c3<\/span>(<\/span>\u2212<\/span>2<\/span>)<\/span><\/span><\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span>=<\/span>1<\/span>+<\/span>7.389<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>=<\/span>8.389<\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>\u2248<\/span>0.119<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

So the sigmoid transformed the enter -2 into roughly 0.119 or 11.9%.<\/p>\n

Now we will see how the sigmoid behaves.<\/p>\n

For a unfavourable enter:<\/p>\n

x=\u22122\u27f6\u03c3(x)\u22480.119x=-2
\nquadlongrightarrowquad
\nsigma(x)approx0.119<\/span>x<\/span>=<\/span><\/span>\u2212<\/span>2<\/span>\u27f6<\/span><\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>\u2248<\/span><\/span>0.119<\/span><\/span><\/span><\/span><\/span><\/div>\n

For zero:<\/p>\n

x=0\u27f6\u03c3(x)=0.5x=0
\nquadlongrightarrowquad
\nsigma(x)=0.5<\/span>x<\/span>=<\/span><\/span>0<\/span>\u27f6<\/span><\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>0.5<\/span><\/span><\/span><\/span><\/span><\/div>\n

For a optimistic enter:<\/p>\n

x=2\u27f6\u03c3(x)\u22480.881x=2
\nquadlongrightarrowquad
\nsigma(x)approx0.881<\/span>x<\/span>=<\/span><\/span>2<\/span>\u27f6<\/span><\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>\u2248<\/span><\/span>0.881<\/span><\/span><\/span><\/span><\/span><\/div>\n

In order x will increase, the sigmoid output strikes from values near 0, passes via 0.5 and strikes towards 1.<\/p>\n

Within the excessive instances:<\/p>\n

x\u2192\u2212\u221e\u27f9\u03c3(x)\u21920xrightarrow-infty
\nquadLongrightarrowquad
\nsigma(x)rightarrow0<\/span>x<\/span>\u2192<\/span><\/span>\u2212<\/span>\u221e<\/span>\u27f9<\/span><\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>\u2192<\/span><\/span>0<\/span><\/span><\/span><\/span><\/span><\/div>\n

and<\/p>\n

x\u2192+\u221e\u27f9\u03c3(x)\u21921xrightarrow+infty
\nquadLongrightarrowquad
\nsigma(x)rightarrow1<\/span>x<\/span>\u2192<\/span><\/span>+<\/span>\u221e<\/span>\u27f9<\/span><\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>\u2192<\/span><\/span>1<\/span><\/span><\/span><\/span><\/span><\/div>\n

That is precisely the habits we needed from a perform that transforms any actual quantity into one thing between 0 and 1.<\/p>\n

\n
\"\"<\/div>
Picture by Creator<\/figcaption><\/figure>\n

Now we’ve got an concept of how we acquired the equation of the sigmoid perform.<\/p>\n

For those who bear in mind, in my latest blogs, after we mentioned backpropagation and neural networks normally, we talked about activation features and why they’re vital.<\/p>\n

We used the ReLU activation perform to know these ideas.<\/p>\n

Now, we will additionally use sigmoid as an activation perform.<\/p>\n

But when we use sigmoid as an activation perform, there may be another factor we have to know.<\/p>\n

Through the backward cross, we already know that the community calculates gradients utilizing derivatives. <\/p>\n

So, if sigmoid is a part of the community, we have to differentiate it as properly.<\/p>\n

Now let’s focus solely on deriving the by-product of the sigmoid perform step-by-step.<\/p>\n

\u03c3(x)=11+e\u2212xsigma(x)=frac{1}{1+e^{-x}}<\/span>\u03c3<\/span>(<\/span>x<\/span>)<\/span>=<\/span><\/span>1<\/span>+<\/span>e<\/span>\u2212<\/span>x<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>1<\/span><\/span><\/span><\/span>\u200b<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/div>\n

As an alternative of carrying the exponential time period all through calculations, we will merely use the sigmoid output itself.<\/p>\n

That is the by-product we use each time sigmoid seems within the gradient calculations of a neural community.<\/p>\n

\u00b7\u00b7\u00b7<\/p>\n

Abstract<\/h3>\n

Within the upcoming blogs, we’re going to focus on subjects like vanishing gradients and exploding gradients.<\/p>\n

As we discover these subjects, we are going to come throughout the sigmoid perform, and we can even want its by-product.<\/p>\n

If we derive the sigmoid perform and its by-product in these blogs, the dialogue might turn into lengthy, and we could lose deal with the precise idea that we are attempting to know. <\/p>\n

It might even be higher to have an concept of the place the sigmoid perform and its by-product come from earlier than utilizing them in additional ideas.<\/p>\n

We first began with the financial institution instance to see how e seems. We then realized about its vital property in calculus and, utilizing these concepts, step by step constructed the sigmoid equation.<\/p>\n

We noticed how this equation is utilized in logistic regression and neural networks, and we additionally derived its by-product.<\/p>\n

Now, after we transfer on to the upcoming subjects, we have already got this basis which can be helpful for us. <\/p>\n

I hope you discovered this weblog useful in understanding an idea that we incessantly use.<\/p>\n

You probably have any questions or options for enchancment, be happy to share them within the feedback on LinkedIn.<\/p>\n

And if you have not learn my newest weblog sequence on backpropagation but, you’ll be able to learn it right here.<\/p>\n

\n

Generally, transferring ahead means going again and understanding the fundamentals.<\/p>\n<\/blockquote>\n

Thanks for studying!<\/p>\n

\u00b7\u00b7\u00b7<\/p>\n<\/div>\n


\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Welcome again! We lately mentioned backpropagation, and I hope you now have an concept of what backpropagation is and the way it really works. Let’s proceed the deep studying journey. Despite the fact that we apply the backpropagation algorithm to a neural community, we nonetheless have some issues, and vanishing gradients is considered one of […]<\/p>\n","protected":false},"author":1,"featured_media":4342,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","fifu_image_alt":"","jnews-multi-image_gallery":[],"jnews_single_post":[],"jnews_primary_category":[],"jnews_override_bookmark_settings":[],"jnews_social_meta":[],"jnews_override_counter":[],"footnotes":""},"categories":[7],"tags":[1740,2977,2359,4557],"class_list":["post-4340","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-science-mlops","tag-function","tag-networks","tag-neural","tag-sigmoid"],"yoast_head":"\nThe Sigmoid Perform: From 'e' to Neural Networks - Future News 24<\/title>\n<meta name=\"description\" content=\"We use the equation all the time. But where did it actually come from?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Sigmoid Perform: From 'e' to Neural Networks - Future News 24\" \/>\n<meta property=\"og:description\" content=\"We use the equation all the time. But where did it actually come from?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/\" \/>\n<meta property=\"og:site_name\" content=\"Future News 24\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-27T18:35:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-08-28T08:59:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\" \/>\n<meta name=\"author\" content=\"Future News 24\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Future News 24\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"19 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/\"},\"author\":{\"name\":\"Future News 24\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/person\\\/cecad1bde21cfc357cf70128144d6c83\"},\"headline\":\"The Sigmoid Perform: From ‘e’ to Neural Networks\",\"datePublished\":\"2026-08-27T18:35:00+00:00\",\"dateModified\":\"2026-08-28T08:59:08+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/\"},\"wordCount\":4108,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/assets.insightmediagroup.io\\\/media\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\",\"keywords\":[\"Function\",\"Networks\",\"Neural\",\"Sigmoid\"],\"articleSection\":[\"Data Science & MLOps\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/\",\"name\":\"The Sigmoid Perform: From 'e' to Neural Networks - Future News 24\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/assets.insightmediagroup.io\\\/media\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\",\"datePublished\":\"2026-08-27T18:35:00+00:00\",\"dateModified\":\"2026-08-28T08:59:08+00:00\",\"description\":\"We use the equation all the time. But where did it actually come from?\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#primaryimage\",\"url\":\"https:\\\/\\\/assets.insightmediagroup.io\\\/media\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\",\"contentUrl\":\"https:\\\/\\\/assets.insightmediagroup.io\\\/media\\\/wp-content\\\/uploads\\\/2026\\\/08\\\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/2026\\\/08\\\/27\\\/the-sigmoid-function-from-e-to-neural-networks\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/futurenews24.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Sigmoid Perform: From ‘e’ to Neural Networks\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#website\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/\",\"name\":\"Future News 24\",\"description\":\"The Smart Hub for AI and Next-Gen Innovation\",\"publisher\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/futurenews24.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#organization\",\"name\":\"Future News 24\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/futurenews24.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/fn24-favicon.png\",\"contentUrl\":\"https:\\\/\\\/futurenews24.com\\\/wp-content\\\/uploads\\\/2026\\\/06\\\/fn24-favicon.png\",\"width\":250,\"height\":250,\"caption\":\"Future News 24\"},\"image\":{\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/futurenews24.com\\\/#\\\/schema\\\/person\\\/cecad1bde21cfc357cf70128144d6c83\",\"name\":\"Future News 24\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g\",\"caption\":\"Future News 24\"},\"sameAs\":[\"https:\\\/\\\/futurenews24.com\"],\"url\":\"https:\\\/\\\/futurenews24.com\\\/index.php\\\/author\\\/mridulpahuja20\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Sigmoid Perform: From 'e' to Neural Networks - Future News 24","description":"We use the equation all the time. But where did it actually come from?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/","og_locale":"en_US","og_type":"article","og_title":"The Sigmoid Perform: From 'e' to Neural Networks - Future News 24","og_description":"We use the equation all the time. But where did it actually come from?","og_url":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/","og_site_name":"Future News 24","article_published_time":"2026-08-27T18:35:00+00:00","article_modified_time":"2026-08-28T08:59:08+00:00","og_image":[{"url":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","type":"","width":"","height":""}],"author":"Future News 24","twitter_card":"summary_large_image","twitter_image":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","twitter_misc":{"Written by":"Future News 24","Est. reading time":"19 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#article","isPartOf":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/"},"author":{"name":"Future News 24","@id":"https:\/\/futurenews24.com\/#\/schema\/person\/cecad1bde21cfc357cf70128144d6c83"},"headline":"The Sigmoid Perform: From ‘e’ to Neural Networks","datePublished":"2026-08-27T18:35:00+00:00","dateModified":"2026-08-28T08:59:08+00:00","mainEntityOfPage":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/"},"wordCount":4108,"commentCount":0,"publisher":{"@id":"https:\/\/futurenews24.com\/#organization"},"image":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#primaryimage"},"thumbnailUrl":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","keywords":["Function","Networks","Neural","Sigmoid"],"articleSection":["Data Science & MLOps"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/","url":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/","name":"The Sigmoid Perform: From 'e' to Neural Networks - Future News 24","isPartOf":{"@id":"https:\/\/futurenews24.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#primaryimage"},"image":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#primaryimage"},"thumbnailUrl":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","datePublished":"2026-08-27T18:35:00+00:00","dateModified":"2026-08-28T08:59:08+00:00","description":"We use the equation all the time. But where did it actually come from?","breadcrumb":{"@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#primaryimage","url":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg","contentUrl":"https:\/\/assets.insightmediagroup.io\/media\/wp-content\/uploads\/2026\/08\/pexels-claudia-schmalz-3928374-6037411-scaled.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/futurenews24.com\/index.php\/2026\/08\/27\/the-sigmoid-function-from-e-to-neural-networks\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/futurenews24.com\/"},{"@type":"ListItem","position":2,"name":"The Sigmoid Perform: From ‘e’ to Neural Networks"}]},{"@type":"WebSite","@id":"https:\/\/futurenews24.com\/#website","url":"https:\/\/futurenews24.com\/","name":"Future News 24","description":"The Smart Hub for AI and Next-Gen Innovation","publisher":{"@id":"https:\/\/futurenews24.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/futurenews24.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/futurenews24.com\/#organization","name":"Future News 24","url":"https:\/\/futurenews24.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/futurenews24.com\/#\/schema\/logo\/image\/","url":"https:\/\/futurenews24.com\/wp-content\/uploads\/2026\/06\/fn24-favicon.png","contentUrl":"https:\/\/futurenews24.com\/wp-content\/uploads\/2026\/06\/fn24-favicon.png","width":250,"height":250,"caption":"Future News 24"},"image":{"@id":"https:\/\/futurenews24.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/futurenews24.com\/#\/schema\/person\/cecad1bde21cfc357cf70128144d6c83","name":"Future News 24","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d57f07142d73cb5503ab2446ea7bc9ef3d0a5ba378d64a6157692311e42bf097?s=96&d=mm&r=g","caption":"Future News 24"},"sameAs":["https:\/\/futurenews24.com"],"url":"https:\/\/futurenews24.com\/index.php\/author\/mridulpahuja20\/"}]}},"_links":{"self":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/4340","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/comments?post=4340"}],"version-history":[{"count":1,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/4340\/revisions"}],"predecessor-version":[{"id":4341,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/posts\/4340\/revisions\/4341"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/media\/4342"}],"wp:attachment":[{"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/media?parent=4340"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/categories?post=4340"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/futurenews24.com\/index.php\/wp-json\/wp\/v2\/tags?post=4340"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}