Dynamic Optimization
Linear Programming Approaches

Z. Wen, L. J. Durlofsky, B. Van Roy, and K. Aziz,
``Approximate
Dynamic Programming for Optimizing Oil Production,''
Chapter 25 in Reinforcement Learning and Approximate Dynamic Programming for
Feedback Control, edited by F. L. Lewis and D. Liu, WileyIEEE Press, 2012.

Z. Wen, L. J. Durlofsky, B. Van Roy, and K. Aziz,
``Use of
Approximate Dynamic Programming for Production Optimization,'' forthcoming in the SPE Proceedings.

J. Han and B. Van Roy, ``Control of
Diffusions via Linear Programming,'' in Stochastic Programming:
The State of the Art, in Honor of George B. Dantzig, edited by Gerd
Infanger, pp. 329354, Springer, 2011.

V. F. Farias and B. Van Roy, ``An
Approximate Dynamic Programming
Approach to Network Revenue Management,'' 2007.

D. P. de Farias and B. Van Roy,
``A
CostShaping Linear Program for AverageCost Approximate Dynamic
Programming with Performance Guarantees,''
Mathematics of Operations Research, Vol. 31, No. 3, pp. 597620,
2006.

R. Cogill, M. Rotkowitz, B. Van Roy, S. Lall,
``An
Approximate Dynamic Programming Approach to Decentralized Control
of Stochastic Systems,''
Lecture Notes in Control and Information Sciences,
Springer, Berlin, 2006, Vol. 329, pp. 243256.

V. F. Farias and B. Van Roy,
``Tetris:
A Study of Randomized Constraint Sampling,''
in Probabilistic and Randomized Methods for Design
Under Uncertainty, G. Calafiore and F. Dabbene, eds., SpringerVerlag,
2006.

D. P. de Farias and B. Van Roy,
``
On Constraint Sampling in the Linear Programming Approach to
Approximate Dynamic Programming,''
Mathematics of Operations Research, Vol. 29, No. 3,
August 2004, pp. 462478.

D. P. de Farias and B. Van Roy,
``The
Linear Programming Approach to Approximate Dynamic Programming,''
Operations Research, Vol. 51, No. 6, NovemberDecember 2003,
pp. 850865.
Approximate Value Iteration and TemporalDifference Methods

B. Van Roy,
``On
RegressionBased Stopping Times,'' Discrete Event
Dynamic Systems, Vol. 20, No. 3, pp. 307324, 2010.

C. C. Moallemi, S. Kumar, and B. Van Roy, ``Approximate
and DataDriven Dynamic
Programming for Queueing Networks,'' 2008.

B. Van Roy
``Performance
Loss Bounds for Approximate Value Iteration with State Aggregation,''
Mathematics of Operations Research, Vol. 31, No. 2, pp. 234244,
2006.

D. S. Choi and B. Van Roy,
``A
Generalized Kalman Filter for Fixed Point Approximation
and Efficient TemporalDifference Learning,''
Discrete Event Dynamic Systems, Vol. 16, No. 2, April 2006.

B. Van Roy, ``
NeuroDynamic Programming: Overview and Recent
Trends,'' in Handbook of Markov Decision
Processes: Methods and Applications,
edited by E. Feinberg and A. Shwartz,
Kluwer, 2001.
 J. N. Tsitsiklis and B. Van Roy,
``
On Average Versus Discounted Reward TemporalDifference
Learning,'' Machine Learning, Vol. 49, No. 23, 2002, pp. 179191.

J. N. Tsitsiklis and B. Van Roy,
``Regression Methods
for Pricing Complex AmericanStyle Options,''
IEEE Transactions on Neural Networks,
Vol. 12, No. 4 (special issue on computational finance), July 2001,
pp. 694703.

D. P. de Farias and B. Van Roy,
``
On the Existence of Fixed Points for Approximate Value
Iteration and TemporalDifference Learning,''
Journal of Optimization Theory and Applications,
Vol. 105, No. 3, June, 2000.
 J. N. Tsitsiklis and B. Van Roy,
``Average Cost
TemporalDifference Learning,'' Automatica,
Vol. 35, No. 11, November 1999, pp. 17991808.
 J. N. Tsitsiklis and B. Van Roy,
``Optimal Stopping of
Markov Processes: Hilbert Space Theory,
Approximation Algorithms, and an
Application to Pricing HighDimensional
Financial Derivatives,''
IEEE Transactions on Automatic Control,
Vol. 44, No. 10, October 1999, pp. 18401851.
 J. N. Tsitsiklis and B. Van Roy,
``An Analysis of
TemporalDifference Learning with Function Approximation,''
IEEE Transactions on Automatic Control,
Vol. 42, No. 5, May 1997, pp. 674690.
 J. N. Tsitsiklis and B. Van Roy, ``FeatureBased Methods for
Large Scale Dynamic Programming,'' Machine
Learning, Vol. 22, 1996, pp. 5994.

B. Van Roy, D. P. Bertsekas, Y. Lee, and J. N. Tsitsiklis,
``A NeuroDynamic Programming Approach to Retailer Inventory
Management,'' Proceedings of the IEEE Conference on
Decision and Control, 1997.
(full length version)
Miscellaneous

H. Permuter, P. Cuff, B. Van Roy, and T. Weissman, ``Capacity of the
Trapdoor Channel with Feedback,'' IEEE
Transactions on Information Theory, Vol. 54, No. 7, pp. 31503165,
2008.

V. F. Farias and B. Van Roy,
``Dynamic
Pricing with a Prior on Market Response,'' Operations Research,
Vol. 58, No. 1, pp. 1629, 2010.

B. Van Roy,
``A Short
Proof of Optimality for the MIN Cache Replacement Algorithm,''
Information Processing Letters, Vol. 102, No. 2, pp. 7273, 2007.

G. Y. Weintraub, C. L. Benkard, and B. Van Roy,
``Computational
Methods for Oblivious Equilibrium,'' Operations Research,
Vol. 58, No. 4, pp. 12471265, 2010.
[Matlab
code (updated July 2012)]

G. Y. Weintraub, L. C. Benkard, and B. Van Roy,
``Markov
Perfect Industry Dynamics with
Many Firms,'' Econometrica, Vol. 76, No. 6, 2008, pp. 13751411.
[Technical Appendix]

V. F. Farias and B. Van Roy
``Approximation
Algorithms for Dynamic Resource Allocation,''
Operations Research Letters, Vol. 34, No. 2, March 2006,
pp. 180190.

X. Yan, P. Diaconis, P. Rusmevichientong, and B. Van Roy,
``Solitaire:
Man Versus Machine,''
Advances in Neural Information Processing Systems 17,
MIT Press, 2005.

C. C. Moallemi and B. Van Roy
``Distributed
Optimization in Adaptive Networks,'' Advances in Neural Information
Processing Systems 16, MIT Press, 2004.
[appendix]

P. Rusmevichientong and B. Van Roy,
``A
Tractable POMDP for a Class of Sequencing Problems,''
Proceedings of the Conference on Uncertainty in Artificial
Intelligence, 2001.

N. O. Keohane, B. Van Roy, and R. J. Zeckhauser,
``Managing
the Quality of a Resource with Stock and Flow Controls,''
Journal of Public Economics, Vol. 91, 2007, pp. 541569.