(IDMO 3, Problem 6) Determine all continuous functions $f:\mathbb{R}\to\mathbb{R}$ such that the set of functions $g:\mathbb R\to\mathbb R$ satisfying $$g(a)f(b)+g(b)f(a)\leqslant (a+f(a))(b+f(b)) \text{ for all }a,b\in\mathbb R$$is finite but nonempty.
Walkthrough. Almost immediately we can figure out that if at least two distinct $g$ exists, then a whole continuum of solutions exists. However, we will soon find out this isn’t particularly useful.
When you’re reading through this, be aware that there’s a long list of things that don’t work (which I didn’t keep track of). The difficulty of the problem boils down to two main things:
The following are the successful lines of thought I had over the course of the problem (again, many things failed, and I really don’t remember any of them):
(1) - After a while, you’ll want to try to solve this inequality for a special case of $f$. One natural $f$ to try is $f(x) =x$. When we set $g(x) = 2x$, equality always holds. $$bg(a) + ag(b)\le 4ab$$
Solving this case is much easier but not completely trivial, because it is still unclear how to deal with the $\le$ sign. Eventually, we find that $a=0$ gets $bg(0)\le 0$, but varying $b\in\mathbb{R}$ we get $g(0) = 0$. With some imagination, we try both $(a,b)$ and $(-a,b)$ and get to $b(g(a) + g(-a))\le 0$, which by the above reasoning gets us $g(a) + g(-a)=0$.
We can generalize this slightly by adding a leading coefficient: if $a>0>c$, then: $$\begin{align*}(-c)(bg(a)+ag(b))&\le -4abc \\ a(bg(c) + cg(b)) &\le 4abc \end{align*}$$ Then we get $cg(a)-ag(c)=0$ and it is easy to finish from here.
Of course, it isn’t much harder to solve this for the general family $f(x) = \lambda x$ where $\lambda\neq 0$.
(2) - Inspired by the success of (1), we try to recreate this in the general case: however, the simultaneous cancelling doesn’t quite work because $(t+f(t), f(t))$ are no longer scalar multiples of each other. We should start to (mildly) suspect that the only $f$ that work should be linear (which actually turns out to be false).
But anyway, we try our best: supposing that $f(a) = 0$: $$g(a)f(b) \le a(b+f(b))$$ or equivalently, $$(g(a) - a)f(b)\le ab$$ Assuming that $g(a)\neq a$, we get some kind of linear bound of the form $f(b)\le \mu b$ (or $f(b)\ge \mu b$, depending on the signs). This isn’t super impressive, but at this point I’m inspired to guess that this might be sufficient to get a solution $g$ (I think I had a vague mental image of $g$ being a linear combination of $x$ and $f(x)$, but I can’t recall any other specifics). Indeed, starting from $$(f(a) - \mu a)(f(b) - \mu b)\ge 0$$ we can expand and get form for $g$ (under some assumptions like $\mu\neq 0$). We also figure that the right form to be working with is $x\ge \mu f(x)$.
(2.5) - Here is a slight (optional?) quality of life improvement inspired by (2). If you shift $g$ by the correct amount, we can work with $h$ where $$h(a)f(b) + h(b)f(a) \le ab$$ which looks a lot friendlier for computations.
(3) - Now for the tour de force: we try to use what we learnt in (2) on the method in (1). If $f(a) = 0$, $$h(a)f(b) \le ab$$ So our $\mu$ is fixed at $h(a) / a$. Admittedly, this isn’t very strong.
Keep going: suppose $f(a)+f(a')=0$, then adding gives: $$f(b)(h(a) + h(a')) \le b(a + a')$$ Not the best either: we can imagine using some kind of intermediate value theorem to get the existence of $a'$, but we run into all kinds of obstructions (e.g. if $f$ is bounded on one side).
At this point we should either ditch this idea, or crank it up to 100. We opt for the latter: suppose $\sum f(a_i) = 0$, then $$f(b)(\sum h(a_i))\le b(\sum a_i)$$ This gets rid of most of the obstructions earlier, and we’re left with the cases $f\ge 0$ or $f\le 0$ (which are not too hard to directly construct). But again, there’s a problem that we still can’t isolate and manipulate individual values of $f(x)$. But we remember the very last thing we did in (1), and take $\sum \lambda_if(a_i)=0$ for positive $\lambda_i>0$. This version is finally strong enough to crack the problem wide open: suppose that $\eta_1(a_1,f(a_1)) + \eta_2(a_2,f(a_2)) + \eta_3(a_3,f(a_3)) = (0,0)$, then if we do a slight adjustment from $$(\lambda_i) = (1,1,...)\rightarrow (1+\varepsilon\eta_1, 1-\varepsilon\eta_2 , 1+\varepsilon\eta_3, 1, ...)$$ then we immediately get $$\sum_{i=1,2,3}\eta_i h(a_i) = 0$$ This means that any triple in $\{(x,f(x),h(x))\}_{x\in \mathbb{R}}$ are linearly dependent, so there exists constants $a,b,c$ where $$ax+bf(x) + ch(x) = 0$$ In particular, for nonlinear $f$, $h$ is forced to be a linear combination of $x$ and $f(x)$. This also means that the value of $h$ is uniquely determined by two points. Write $h(x) = \mu_1 x + \mu_2f(x)$ then $$\mu_1af(b) + \mu_1bf(a) + 2\mu_2f(a)f(b) \le ab$$ $$\Leftrightarrow (\mu_1a + 2\mu_2 f(a))f(b)\le (a-\mu_1f(a))b $$ With the additional assumption that $\mu = \frac{\mu_1a+2\mu_2f(a)}{a-\mu_1f(a)}$ must be unique and $f$ is non-linear, we get that $\mu_1,\mu_2$ are uniquely determined. $\square$
Answer. All $f$ such that there exists a unique $\mu\neq 0$ such that $f(x) \ge \mu x$ or $f(x) \le \mu x$ (whaaaaat).